=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-490121 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-490121 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m8.427415912s)
-- stdout --
* [old-k8s-version-490121] minikube v1.32.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18239
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18239-558171/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18239-558171/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.28.4 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.28.4
* Using the docker driver based on existing profile
* Starting "old-k8s-version-490121" primary control-plane node in "old-k8s-version-490121" cluster
* Pulling base image v0.0.42-1708944392-18244 ...
* Restarting existing docker container for "old-k8s-version-490121" ...
* Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-490121 addons enable metrics-server
* Enabled addons: default-storageclass, dashboard, storage-provisioner, metrics-server
-- /stdout --
** stderr **
I0307 19:30:26.308206 762831 out.go:291] Setting OutFile to fd 1 ...
I0307 19:30:26.308868 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:30:26.308883 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:30:26.308889 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:30:26.309246 762831 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18239-558171/.minikube/bin
I0307 19:30:26.309723 762831 out.go:298] Setting JSON to false
I0307 19:30:26.310875 762831 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":11570,"bootTime":1709828256,"procs":203,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0307 19:30:26.310949 762831 start.go:139] virtualization:
I0307 19:30:26.313810 762831 out.go:177] * [old-k8s-version-490121] minikube v1.32.0 on Ubuntu 20.04 (arm64)
I0307 19:30:26.316202 762831 out.go:177] - MINIKUBE_LOCATION=18239
I0307 19:30:26.318032 762831 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0307 19:30:26.316244 762831 notify.go:220] Checking for updates...
I0307 19:30:26.319861 762831 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18239-558171/kubeconfig
I0307 19:30:26.321669 762831 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18239-558171/.minikube
I0307 19:30:26.323603 762831 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0307 19:30:26.325594 762831 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0307 19:30:26.328419 762831 config.go:182] Loaded profile config "old-k8s-version-490121": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0307 19:30:26.331073 762831 out.go:177] * Kubernetes 1.28.4 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.28.4
I0307 19:30:26.332939 762831 driver.go:392] Setting default libvirt URI to qemu:///system
I0307 19:30:26.354981 762831 docker.go:122] docker version: linux-25.0.4:Docker Engine - Community
I0307 19:30:26.355101 762831 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0307 19:30:26.425683 762831 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-07 19:30:26.415773364 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0307 19:30:26.425793 762831 docker.go:295] overlay module found
I0307 19:30:26.428005 762831 out.go:177] * Using the docker driver based on existing profile
I0307 19:30:26.430936 762831 start.go:297] selected driver: docker
I0307 19:30:26.430956 762831 start.go:901] validating driver "docker" against &{Name:old-k8s-version-490121 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-490121 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0307 19:30:26.431065 762831 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0307 19:30:26.431748 762831 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0307 19:30:26.493672 762831 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-07 19:30:26.484540586 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0307 19:30:26.494024 762831 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0307 19:30:26.494087 762831 cni.go:84] Creating CNI manager for ""
I0307 19:30:26.494104 762831 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0307 19:30:26.494163 762831 start.go:340] cluster config:
{Name:old-k8s-version-490121 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-490121 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0307 19:30:26.496588 762831 out.go:177] * Starting "old-k8s-version-490121" primary control-plane node in "old-k8s-version-490121" cluster
I0307 19:30:26.498341 762831 cache.go:121] Beginning downloading kic base image for docker with containerd
I0307 19:30:26.499986 762831 out.go:177] * Pulling base image v0.0.42-1708944392-18244 ...
I0307 19:30:26.501685 762831 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0307 19:30:26.501744 762831 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0307 19:30:26.501758 762831 cache.go:56] Caching tarball of preloaded images
I0307 19:30:26.501769 762831 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon
I0307 19:30:26.501838 762831 preload.go:173] Found /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0307 19:30:26.501848 762831 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0307 19:30:26.501968 762831 profile.go:142] Saving config to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/config.json ...
I0307 19:30:26.519503 762831 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon, skipping pull
I0307 19:30:26.519527 762831 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in daemon, skipping load
I0307 19:30:26.519565 762831 cache.go:194] Successfully downloaded all kic artifacts
I0307 19:30:26.519594 762831 start.go:360] acquireMachinesLock for old-k8s-version-490121: {Name:mk7022b7081856386136ba228ad21410889c3d49 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0307 19:30:26.519665 762831 start.go:364] duration metric: took 43.7µs to acquireMachinesLock for "old-k8s-version-490121"
I0307 19:30:26.519685 762831 start.go:96] Skipping create...Using existing machine configuration
I0307 19:30:26.519690 762831 fix.go:54] fixHost starting:
I0307 19:30:26.519975 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:26.535978 762831 fix.go:112] recreateIfNeeded on old-k8s-version-490121: state=Stopped err=<nil>
W0307 19:30:26.536016 762831 fix.go:138] unexpected machine state, will restart: <nil>
I0307 19:30:26.538448 762831 out.go:177] * Restarting existing docker container for "old-k8s-version-490121" ...
I0307 19:30:26.540252 762831 cli_runner.go:164] Run: docker start old-k8s-version-490121
I0307 19:30:26.841652 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:26.864693 762831 kic.go:430] container "old-k8s-version-490121" state is running.
I0307 19:30:26.865103 762831 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-490121
I0307 19:30:26.891392 762831 profile.go:142] Saving config to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/config.json ...
I0307 19:30:26.891640 762831 machine.go:94] provisionDockerMachine start ...
I0307 19:30:26.891706 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:26.912591 762831 main.go:141] libmachine: Using SSH client type: native
I0307 19:30:26.912863 762831 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33808 <nil> <nil>}
I0307 19:30:26.912873 762831 main.go:141] libmachine: About to run SSH command:
hostname
I0307 19:30:26.913483 762831 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:36758->127.0.0.1:33808: read: connection reset by peer
I0307 19:30:30.050086 762831 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-490121
I0307 19:30:30.050129 762831 ubuntu.go:169] provisioning hostname "old-k8s-version-490121"
I0307 19:30:30.050205 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:30.072173 762831 main.go:141] libmachine: Using SSH client type: native
I0307 19:30:30.072464 762831 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33808 <nil> <nil>}
I0307 19:30:30.072484 762831 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-490121 && echo "old-k8s-version-490121" | sudo tee /etc/hostname
I0307 19:30:30.224347 762831 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-490121
I0307 19:30:30.224434 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:30.243257 762831 main.go:141] libmachine: Using SSH client type: native
I0307 19:30:30.243518 762831 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33808 <nil> <nil>}
I0307 19:30:30.243542 762831 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-490121' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-490121/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-490121' | sudo tee -a /etc/hosts;
fi
fi
I0307 19:30:30.380423 762831 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0307 19:30:30.380451 762831 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18239-558171/.minikube CaCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18239-558171/.minikube}
I0307 19:30:30.380472 762831 ubuntu.go:177] setting up certificates
I0307 19:30:30.380481 762831 provision.go:84] configureAuth start
I0307 19:30:30.380541 762831 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-490121
I0307 19:30:30.408378 762831 provision.go:143] copyHostCerts
I0307 19:30:30.408446 762831 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem, removing ...
I0307 19:30:30.408460 762831 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem
I0307 19:30:30.408540 762831 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem (1082 bytes)
I0307 19:30:30.408687 762831 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem, removing ...
I0307 19:30:30.408693 762831 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem
I0307 19:30:30.408721 762831 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem (1123 bytes)
I0307 19:30:30.408787 762831 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem, removing ...
I0307 19:30:30.408792 762831 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem
I0307 19:30:30.408817 762831 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem (1675 bytes)
I0307 19:30:30.408869 762831 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-490121 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-490121]
I0307 19:30:31.200107 762831 provision.go:177] copyRemoteCerts
I0307 19:30:31.200192 762831 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0307 19:30:31.200255 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:31.229660 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:31.326822 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0307 19:30:31.356281 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0307 19:30:31.391623 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0307 19:30:31.424440 762831 provision.go:87] duration metric: took 1.043933521s to configureAuth
I0307 19:30:31.424518 762831 ubuntu.go:193] setting minikube options for container-runtime
I0307 19:30:31.424778 762831 config.go:182] Loaded profile config "old-k8s-version-490121": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0307 19:30:31.424809 762831 machine.go:97] duration metric: took 4.533152343s to provisionDockerMachine
I0307 19:30:31.424840 762831 start.go:293] postStartSetup for "old-k8s-version-490121" (driver="docker")
I0307 19:30:31.424866 762831 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0307 19:30:31.424978 762831 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0307 19:30:31.425051 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:31.453213 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:31.552317 762831 ssh_runner.go:195] Run: cat /etc/os-release
I0307 19:30:31.557096 762831 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0307 19:30:31.557129 762831 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0307 19:30:31.557138 762831 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0307 19:30:31.557146 762831 info.go:137] Remote host: Ubuntu 22.04.3 LTS
I0307 19:30:31.557156 762831 filesync.go:126] Scanning /home/jenkins/minikube-integration/18239-558171/.minikube/addons for local assets ...
I0307 19:30:31.557213 762831 filesync.go:126] Scanning /home/jenkins/minikube-integration/18239-558171/.minikube/files for local assets ...
I0307 19:30:31.557290 762831 filesync.go:149] local asset: /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem -> 5635812.pem in /etc/ssl/certs
I0307 19:30:31.557400 762831 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0307 19:30:31.566574 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem --> /etc/ssl/certs/5635812.pem (1708 bytes)
I0307 19:30:31.602504 762831 start.go:296] duration metric: took 177.635747ms for postStartSetup
I0307 19:30:31.602585 762831 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0307 19:30:31.602639 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:31.621032 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:31.711600 762831 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0307 19:30:31.716908 762831 fix.go:56] duration metric: took 5.197210488s for fixHost
I0307 19:30:31.716939 762831 start.go:83] releasing machines lock for "old-k8s-version-490121", held for 5.197266299s
I0307 19:30:31.717009 762831 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-490121
I0307 19:30:31.762500 762831 ssh_runner.go:195] Run: cat /version.json
I0307 19:30:31.762563 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:31.762797 762831 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0307 19:30:31.762860 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:31.788634 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:31.813455 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:31.886466 762831 ssh_runner.go:195] Run: systemctl --version
I0307 19:30:32.054630 762831 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0307 19:30:32.061076 762831 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0307 19:30:32.099255 762831 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0307 19:30:32.099340 762831 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0307 19:30:32.120116 762831 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0307 19:30:32.120139 762831 start.go:494] detecting cgroup driver to use...
I0307 19:30:32.120173 762831 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0307 19:30:32.120232 762831 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0307 19:30:32.150974 762831 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0307 19:30:32.175790 762831 docker.go:217] disabling cri-docker service (if available) ...
I0307 19:30:32.175849 762831 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0307 19:30:32.195370 762831 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0307 19:30:32.218055 762831 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0307 19:30:32.339339 762831 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0307 19:30:32.469084 762831 docker.go:233] disabling docker service ...
I0307 19:30:32.469205 762831 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0307 19:30:32.489612 762831 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0307 19:30:32.516989 762831 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0307 19:30:32.647378 762831 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0307 19:30:32.775912 762831 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0307 19:30:32.788961 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0307 19:30:32.807628 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0307 19:30:32.819629 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0307 19:30:32.830436 762831 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0307 19:30:32.830567 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0307 19:30:32.841963 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0307 19:30:32.854833 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0307 19:30:32.864735 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0307 19:30:32.878904 762831 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0307 19:30:32.889406 762831 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0307 19:30:32.900317 762831 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0307 19:30:32.910072 762831 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0307 19:30:32.918843 762831 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:30:33.021620 762831 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0307 19:30:33.211462 762831 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0307 19:30:33.211531 762831 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0307 19:30:33.215635 762831 start.go:562] Will wait 60s for crictl version
I0307 19:30:33.215748 762831 ssh_runner.go:195] Run: which crictl
I0307 19:30:33.220134 762831 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0307 19:30:33.311969 762831 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0307 19:30:33.312031 762831 ssh_runner.go:195] Run: containerd --version
I0307 19:30:33.356212 762831 ssh_runner.go:195] Run: containerd --version
I0307 19:30:33.406791 762831 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
I0307 19:30:33.411626 762831 cli_runner.go:164] Run: docker network inspect old-k8s-version-490121 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0307 19:30:33.440324 762831 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0307 19:30:33.444244 762831 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0307 19:30:33.463033 762831 kubeadm.go:877] updating cluster {Name:old-k8s-version-490121 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-490121 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0307 19:30:33.463159 762831 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0307 19:30:33.463234 762831 ssh_runner.go:195] Run: sudo crictl images --output json
I0307 19:30:33.527045 762831 containerd.go:612] all images are preloaded for containerd runtime.
I0307 19:30:33.527076 762831 containerd.go:519] Images already preloaded, skipping extraction
I0307 19:30:33.527137 762831 ssh_runner.go:195] Run: sudo crictl images --output json
I0307 19:30:33.600715 762831 containerd.go:612] all images are preloaded for containerd runtime.
I0307 19:30:33.600741 762831 cache_images.go:84] Images are preloaded, skipping loading
I0307 19:30:33.600750 762831 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.20.0 containerd true true} ...
I0307 19:30:33.600869 762831 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-490121 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-490121 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0307 19:30:33.600940 762831 ssh_runner.go:195] Run: sudo crictl info
I0307 19:30:33.647294 762831 cni.go:84] Creating CNI manager for ""
I0307 19:30:33.647321 762831 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0307 19:30:33.647331 762831 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0307 19:30:33.647388 762831 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-490121 NodeName:old-k8s-version-490121 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0307 19:30:33.647587 762831 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-490121"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0307 19:30:33.647695 762831 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0307 19:30:33.659871 762831 binaries.go:44] Found k8s binaries, skipping transfer
I0307 19:30:33.659944 762831 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0307 19:30:33.669457 762831 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0307 19:30:33.690822 762831 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0307 19:30:33.716800 762831 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0307 19:30:33.741018 762831 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0307 19:30:33.757593 762831 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0307 19:30:33.777426 762831 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:30:33.874617 762831 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0307 19:30:33.891273 762831 certs.go:68] Setting up /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121 for IP: 192.168.85.2
I0307 19:30:33.891298 762831 certs.go:194] generating shared ca certs ...
I0307 19:30:33.891315 762831 certs.go:226] acquiring lock for ca certs: {Name:mke14792b1616e9503645c7147aed38043ea5d64 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:30:33.891449 762831 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18239-558171/.minikube/ca.key
I0307 19:30:33.891505 762831 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.key
I0307 19:30:33.891516 762831 certs.go:256] generating profile certs ...
I0307 19:30:33.891607 762831 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/client.key
I0307 19:30:33.891674 762831 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/apiserver.key.6fe806f2
I0307 19:30:33.891717 762831 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/proxy-client.key
I0307 19:30:33.891823 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581.pem (1338 bytes)
W0307 19:30:33.891860 762831 certs.go:480] ignoring /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581_empty.pem, impossibly tiny 0 bytes
I0307 19:30:33.891873 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem (1675 bytes)
I0307 19:30:33.891896 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem (1082 bytes)
I0307 19:30:33.891921 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem (1123 bytes)
I0307 19:30:33.891945 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem (1675 bytes)
I0307 19:30:33.891990 762831 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem (1708 bytes)
I0307 19:30:33.892618 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0307 19:30:33.919224 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0307 19:30:33.945666 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0307 19:30:33.972650 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0307 19:30:34.002531 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0307 19:30:34.037093 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0307 19:30:34.063237 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0307 19:30:34.088859 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/old-k8s-version-490121/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0307 19:30:34.113766 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581.pem --> /usr/share/ca-certificates/563581.pem (1338 bytes)
I0307 19:30:34.137268 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem --> /usr/share/ca-certificates/5635812.pem (1708 bytes)
I0307 19:30:34.161235 762831 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0307 19:30:34.185068 762831 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0307 19:30:34.204155 762831 ssh_runner.go:195] Run: openssl version
I0307 19:30:34.209838 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/563581.pem && ln -fs /usr/share/ca-certificates/563581.pem /etc/ssl/certs/563581.pem"
I0307 19:30:34.219223 762831 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/563581.pem
I0307 19:30:34.222821 762831 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 7 18:50 /usr/share/ca-certificates/563581.pem
I0307 19:30:34.222891 762831 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/563581.pem
I0307 19:30:34.229956 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/563581.pem /etc/ssl/certs/51391683.0"
I0307 19:30:34.238920 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/5635812.pem && ln -fs /usr/share/ca-certificates/5635812.pem /etc/ssl/certs/5635812.pem"
I0307 19:30:34.248401 762831 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/5635812.pem
I0307 19:30:34.252130 762831 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 7 18:50 /usr/share/ca-certificates/5635812.pem
I0307 19:30:34.252234 762831 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/5635812.pem
I0307 19:30:34.259310 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/5635812.pem /etc/ssl/certs/3ec20f2e.0"
I0307 19:30:34.268499 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0307 19:30:34.277733 762831 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0307 19:30:34.281018 762831 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 7 18:44 /usr/share/ca-certificates/minikubeCA.pem
I0307 19:30:34.281080 762831 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0307 19:30:34.288488 762831 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0307 19:30:34.297236 762831 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0307 19:30:34.300722 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0307 19:30:34.307415 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0307 19:30:34.314082 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0307 19:30:34.320828 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0307 19:30:34.328193 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0307 19:30:34.335113 762831 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0307 19:30:34.341908 762831 kubeadm.go:391] StartCluster: {Name:old-k8s-version-490121 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-490121 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0307 19:30:34.342018 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0307 19:30:34.342097 762831 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0307 19:30:34.379172 762831 cri.go:89] found id: "f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:30:34.379199 762831 cri.go:89] found id: "d69ea4ee77885ba5ed1fabc9e7171fddcad8c266310d6312c0d16e03ed43c9af"
I0307 19:30:34.379204 762831 cri.go:89] found id: "5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:30:34.379207 762831 cri.go:89] found id: "1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:30:34.379210 762831 cri.go:89] found id: "99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:30:34.379213 762831 cri.go:89] found id: "156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:30:34.379216 762831 cri.go:89] found id: "f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:30:34.379255 762831 cri.go:89] found id: "05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:30:34.379266 762831 cri.go:89] found id: ""
I0307 19:30:34.379334 762831 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0307 19:30:34.391630 762831 cri.go:116] JSON = null
W0307 19:30:34.391698 762831 kubeadm.go:398] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0307 19:30:34.391778 762831 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0307 19:30:34.400538 762831 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0307 19:30:34.400560 762831 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0307 19:30:34.400566 762831 kubeadm.go:587] restartPrimaryControlPlane start ...
I0307 19:30:34.400628 762831 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0307 19:30:34.408837 762831 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0307 19:30:34.409433 762831 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-490121" does not appear in /home/jenkins/minikube-integration/18239-558171/kubeconfig
I0307 19:30:34.409711 762831 kubeconfig.go:62] /home/jenkins/minikube-integration/18239-558171/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-490121" cluster setting kubeconfig missing "old-k8s-version-490121" context setting]
I0307 19:30:34.410238 762831 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/kubeconfig: {Name:mk6862a934ece36327360ff645a33ee6e04a2f89 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:30:34.411577 762831 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0307 19:30:34.420500 762831 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.85.2
I0307 19:30:34.420533 762831 kubeadm.go:591] duration metric: took 19.962332ms to restartPrimaryControlPlane
I0307 19:30:34.420543 762831 kubeadm.go:393] duration metric: took 78.645078ms to StartCluster
I0307 19:30:34.420558 762831 settings.go:142] acquiring lock: {Name:mkebfa804b6349436c6d99572f0f0da9cb5ad1f6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:30:34.420614 762831 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18239-558171/kubeconfig
I0307 19:30:34.421497 762831 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/kubeconfig: {Name:mk6862a934ece36327360ff645a33ee6e04a2f89 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:30:34.421725 762831 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0307 19:30:34.425015 762831 out.go:177] * Verifying Kubernetes components...
I0307 19:30:34.422035 762831 config.go:182] Loaded profile config "old-k8s-version-490121": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0307 19:30:34.422059 762831 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0307 19:30:34.427126 762831 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-490121"
I0307 19:30:34.427163 762831 addons.go:69] Setting dashboard=true in profile "old-k8s-version-490121"
I0307 19:30:34.427222 762831 addons.go:234] Setting addon dashboard=true in "old-k8s-version-490121"
W0307 19:30:34.427263 762831 addons.go:243] addon dashboard should already be in state true
I0307 19:30:34.427169 762831 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-490121"
W0307 19:30:34.427307 762831 addons.go:243] addon storage-provisioner should already be in state true
I0307 19:30:34.427329 762831 host.go:66] Checking if "old-k8s-version-490121" exists ...
I0307 19:30:34.427333 762831 host.go:66] Checking if "old-k8s-version-490121" exists ...
I0307 19:30:34.427791 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:34.427850 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:34.427175 762831 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-490121"
I0307 19:30:34.431603 762831 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-490121"
I0307 19:30:34.427236 762831 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:30:34.427184 762831 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-490121"
I0307 19:30:34.431958 762831 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-490121"
W0307 19:30:34.432045 762831 addons.go:243] addon metrics-server should already be in state true
I0307 19:30:34.432081 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:34.432112 762831 host.go:66] Checking if "old-k8s-version-490121" exists ...
I0307 19:30:34.432615 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:34.460631 762831 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0307 19:30:34.462639 762831 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:30:34.462654 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0307 19:30:34.462731 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:34.485040 762831 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0307 19:30:34.484046 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:34.493699 762831 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0307 19:30:34.499922 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0307 19:30:34.499952 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0307 19:30:34.500027 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:34.536624 762831 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-490121"
W0307 19:30:34.536647 762831 addons.go:243] addon default-storageclass should already be in state true
I0307 19:30:34.536672 762831 host.go:66] Checking if "old-k8s-version-490121" exists ...
I0307 19:30:34.537092 762831 cli_runner.go:164] Run: docker container inspect old-k8s-version-490121 --format={{.State.Status}}
I0307 19:30:34.540041 762831 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0307 19:30:34.542182 762831 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0307 19:30:34.540003 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:34.542542 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0307 19:30:34.542599 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:34.574951 762831 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0307 19:30:34.574972 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0307 19:30:34.575036 762831 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-490121
I0307 19:30:34.590718 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:34.613972 762831 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33808 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/old-k8s-version-490121/id_rsa Username:docker}
I0307 19:30:34.626191 762831 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0307 19:30:34.647083 762831 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-490121" to be "Ready" ...
I0307 19:30:34.668612 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:30:34.733023 762831 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0307 19:30:34.733087 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0307 19:30:34.755646 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0307 19:30:34.755712 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0307 19:30:34.770774 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:34.784847 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.784925 762831 retry.go:31] will retry after 284.271879ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.786673 762831 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0307 19:30:34.786730 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0307 19:30:34.802013 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0307 19:30:34.802091 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0307 19:30:34.830230 762831 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0307 19:30:34.830293 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0307 19:30:34.838300 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0307 19:30:34.838372 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0307 19:30:34.864520 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0307 19:30:34.864591 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0307 19:30:34.877838 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0307 19:30:34.896591 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0307 19:30:34.896661 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0307 19:30:34.910036 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.910135 762831 retry.go:31] will retry after 309.047225ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.920660 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0307 19:30:34.920731 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0307 19:30:34.940244 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0307 19:30:34.940318 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0307 19:30:34.959745 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0307 19:30:34.959818 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0307 19:30:34.978844 762831 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:34.978917 762831 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
W0307 19:30:34.987561 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.987635 762831 retry.go:31] will retry after 311.068002ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:34.999407 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:35.069680 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0307 19:30:35.077051 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.077167 762831 retry.go:31] will retry after 259.227355ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:35.145333 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.145419 762831 retry.go:31] will retry after 529.510369ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.220251 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:35.296798 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.296831 762831 retry.go:31] will retry after 298.117273ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.299083 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0307 19:30:35.337231 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0307 19:30:35.385891 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.385929 762831 retry.go:31] will retry after 453.619543ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:35.420318 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.420352 762831 retry.go:31] will retry after 395.650002ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.595668 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:35.669136 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.669168 762831 retry.go:31] will retry after 794.401064ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.675465 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0307 19:30:35.756359 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.756396 762831 retry.go:31] will retry after 650.068344ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.816606 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:35.840100 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:35.900042 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.900092 762831 retry.go:31] will retry after 500.476996ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:35.929733 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:35.929771 762831 retry.go:31] will retry after 746.404218ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.401199 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:36.407568 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:30:36.464333 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:36.497995 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.498073 762831 retry.go:31] will retry after 1.041451192s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:36.525942 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.526015 762831 retry.go:31] will retry after 1.042231632s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:36.567190 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.567224 762831 retry.go:31] will retry after 879.183244ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.647799 762831 node_ready.go:53] error getting node "old-k8s-version-490121": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-490121": dial tcp 192.168.85.2:8443: connect: connection refused
I0307 19:30:36.677162 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:36.746359 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:36.746389 762831 retry.go:31] will retry after 1.136746297s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.446727 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:37.517344 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.517381 762831 retry.go:31] will retry after 1.890351475s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.540701 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:37.569151 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0307 19:30:37.635723 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.635756 762831 retry.go:31] will retry after 1.253904816s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0307 19:30:37.659976 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.660012 762831 retry.go:31] will retry after 1.579356746s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.883379 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:37.960038 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:37.960077 762831 retry.go:31] will retry after 1.014782741s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:38.890127 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0307 19:30:38.963617 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:38.963655 762831 retry.go:31] will retry after 1.977388025s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:38.975852 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:39.070363 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:39.070409 762831 retry.go:31] will retry after 1.693213296s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:39.147924 762831 node_ready.go:53] error getting node "old-k8s-version-490121": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-490121": dial tcp 192.168.85.2:8443: connect: connection refused
I0307 19:30:39.240327 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0307 19:30:39.310809 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:39.310842 762831 retry.go:31] will retry after 2.63520914s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:39.407993 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:39.477145 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:39.477178 762831 retry.go:31] will retry after 2.56162971s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:40.763848 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:40.835373 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:40.835416 762831 retry.go:31] will retry after 1.575887995s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:40.941863 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0307 19:30:41.011738 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:41.011779 762831 retry.go:31] will retry after 3.868888502s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:41.148338 762831 node_ready.go:53] error getting node "old-k8s-version-490121": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-490121": dial tcp 192.168.85.2:8443: connect: connection refused
I0307 19:30:41.946836 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0307 19:30:42.030981 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:42.031016 762831 retry.go:31] will retry after 2.877833945s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:42.039154 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0307 19:30:42.126847 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:42.126891 762831 retry.go:31] will retry after 2.634958605s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:42.411984 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0307 19:30:42.491878 762831 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:42.491914 762831 retry.go:31] will retry after 3.673361459s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0307 19:30:44.762992 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0307 19:30:44.881271 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0307 19:30:44.909286 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:30:46.166446 762831 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0307 19:30:51.291815 762831 node_ready.go:49] node "old-k8s-version-490121" has status "Ready":"True"
I0307 19:30:51.291837 762831 node_ready.go:38] duration metric: took 16.644720969s for node "old-k8s-version-490121" to be "Ready" ...
I0307 19:30:51.291846 762831 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0307 19:30:51.547842 762831 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-mskrg" in "kube-system" namespace to be "Ready" ...
I0307 19:30:51.913425 762831 pod_ready.go:92] pod "coredns-74ff55c5b-mskrg" in "kube-system" namespace has status "Ready":"True"
I0307 19:30:51.913493 762831 pod_ready.go:81] duration metric: took 365.551333ms for pod "coredns-74ff55c5b-mskrg" in "kube-system" namespace to be "Ready" ...
I0307 19:30:51.913541 762831 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:30:52.071583 762831 pod_ready.go:92] pod "etcd-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"True"
I0307 19:30:52.071655 762831 pod_ready.go:81] duration metric: took 158.08547ms for pod "etcd-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:30:52.071684 762831 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:30:53.171170 762831 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (8.408120686s)
I0307 19:30:54.097224 762831 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:30:54.098051 762831 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (9.216739018s)
I0307 19:30:54.100447 762831 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-490121 addons enable metrics-server
I0307 19:30:54.098317 762831 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (9.188984252s)
I0307 19:30:54.098378 762831 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.931899572s)
I0307 19:30:54.102431 762831 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-490121"
I0307 19:30:54.104584 762831 out.go:177] * Enabled addons: default-storageclass, dashboard, storage-provisioner, metrics-server
I0307 19:30:54.106656 762831 addons.go:505] duration metric: took 19.684590078s for enable addons: enabled=[default-storageclass dashboard storage-provisioner metrics-server]
I0307 19:30:56.581498 762831 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:30:59.079158 762831 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:00.081733 762831 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"True"
I0307 19:31:00.081780 762831 pod_ready.go:81] duration metric: took 8.010068986s for pod "kube-apiserver-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:31:00.081794 762831 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:31:02.090160 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:04.588080 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:06.588223 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:08.588823 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:11.088492 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:13.089228 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:15.092115 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:17.093454 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:19.094067 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:21.588252 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:24.088898 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:26.589006 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:29.087601 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:31.089043 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:33.588096 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:35.589059 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:38.088700 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:40.089283 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:42.090608 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:44.588990 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:47.089266 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:49.588650 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:52.087849 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:54.088593 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:56.588309 762831 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:31:57.587908 762831 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"True"
I0307 19:31:57.587936 762831 pod_ready.go:81] duration metric: took 57.506133731s for pod "kube-controller-manager-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:31:57.587949 762831 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-5rbpn" in "kube-system" namespace to be "Ready" ...
I0307 19:31:57.592898 762831 pod_ready.go:92] pod "kube-proxy-5rbpn" in "kube-system" namespace has status "Ready":"True"
I0307 19:31:57.592925 762831 pod_ready.go:81] duration metric: took 4.96872ms for pod "kube-proxy-5rbpn" in "kube-system" namespace to be "Ready" ...
I0307 19:31:57.592936 762831 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:31:59.607148 762831 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:02.098837 762831 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:04.099378 762831 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:06.100007 762831 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:08.599104 762831 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:10.599654 762831 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace has status "Ready":"True"
I0307 19:32:10.599678 762831 pod_ready.go:81] duration metric: took 13.006734482s for pod "kube-scheduler-old-k8s-version-490121" in "kube-system" namespace to be "Ready" ...
I0307 19:32:10.599690 762831 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace to be "Ready" ...
I0307 19:32:12.605589 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:14.606100 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:16.606568 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:19.106001 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:21.106162 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:23.106502 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:25.106995 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:27.606264 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:29.606926 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:32.106769 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:34.606166 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:37.107256 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:39.112526 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:41.605029 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:43.621991 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:46.105021 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:48.106406 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:50.107615 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:52.605407 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:54.605985 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:57.106869 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:32:59.107540 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:01.606304 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:04.106187 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:06.606584 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:09.124646 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:11.605664 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:13.605938 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:15.606165 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:18.106309 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:20.106544 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:22.605997 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:24.606936 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:27.107041 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:29.111031 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:31.606389 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:33.611702 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:36.106153 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:38.607187 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:41.105924 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:43.110415 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:45.605585 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:47.606012 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:49.606708 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:52.106231 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:54.106306 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:56.605836 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:33:58.606251 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:00.607403 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:03.105634 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:05.107060 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:07.606150 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:09.607765 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:12.107152 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:14.606558 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:17.106973 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:19.110832 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:21.606267 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:24.106381 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:26.605921 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:28.606401 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:31.107043 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:33.606534 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:36.106511 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:38.605158 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:40.605737 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:42.607136 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:45.109023 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:47.606200 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:49.608128 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:52.106697 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:54.605489 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:57.105694 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:34:59.109093 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:01.113425 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:03.606877 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:06.107170 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:08.605434 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:10.606286 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:12.608255 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:15.107192 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:17.115034 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:19.606945 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:22.106382 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:24.606733 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:26.607608 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:29.110754 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:31.606730 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:33.656753 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:36.129051 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:38.606994 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:41.105989 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:43.106627 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:45.115823 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:47.609214 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:50.109551 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:52.606769 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:54.611278 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:57.107679 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:59.110902 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:01.134439 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:03.606584 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:05.614979 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:08.112034 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:10.606231 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:10.606263 762831 pod_ready.go:81] duration metric: took 4m0.006564846s for pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace to be "Ready" ...
E0307 19:36:10.606274 762831 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0307 19:36:10.606283 762831 pod_ready.go:38] duration metric: took 5m19.314418834s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0307 19:36:10.606296 762831 api_server.go:52] waiting for apiserver process to appear ...
I0307 19:36:10.606341 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0307 19:36:10.606408 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0307 19:36:10.656033 762831 cri.go:89] found id: "ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:10.656094 762831 cri.go:89] found id: "99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:10.656106 762831 cri.go:89] found id: ""
I0307 19:36:10.656114 762831 logs.go:276] 2 containers: [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939]
I0307 19:36:10.656170 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.659898 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.663752 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0307 19:36:10.663849 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0307 19:36:10.711143 762831 cri.go:89] found id: "5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:10.711195 762831 cri.go:89] found id: "05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:10.711214 762831 cri.go:89] found id: ""
I0307 19:36:10.711239 762831 logs.go:276] 2 containers: [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190]
I0307 19:36:10.711326 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.715686 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.719890 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0307 19:36:10.720008 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0307 19:36:10.782775 762831 cri.go:89] found id: "9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:10.782850 762831 cri.go:89] found id: "f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:10.782869 762831 cri.go:89] found id: ""
I0307 19:36:10.782892 762831 logs.go:276] 2 containers: [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384]
I0307 19:36:10.782980 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.787225 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.791833 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0307 19:36:10.791972 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0307 19:36:10.850718 762831 cri.go:89] found id: "62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:10.850794 762831 cri.go:89] found id: "156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:10.850813 762831 cri.go:89] found id: ""
I0307 19:36:10.850837 762831 logs.go:276] 2 containers: [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4]
I0307 19:36:10.850923 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.854985 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.858671 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0307 19:36:10.858755 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0307 19:36:10.909124 762831 cri.go:89] found id: "eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:10.909149 762831 cri.go:89] found id: "5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:10.909155 762831 cri.go:89] found id: ""
I0307 19:36:10.909162 762831 logs.go:276] 2 containers: [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907]
I0307 19:36:10.909233 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.913385 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.917239 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0307 19:36:10.917311 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0307 19:36:10.962456 762831 cri.go:89] found id: "2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:10.962475 762831 cri.go:89] found id: "f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:10.962480 762831 cri.go:89] found id: ""
I0307 19:36:10.962487 762831 logs.go:276] 2 containers: [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08]
I0307 19:36:10.962547 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.966970 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.970855 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0307 19:36:10.970976 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0307 19:36:11.016102 762831 cri.go:89] found id: "18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:11.016128 762831 cri.go:89] found id: "1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:11.016134 762831 cri.go:89] found id: ""
I0307 19:36:11.016141 762831 logs.go:276] 2 containers: [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29]
I0307 19:36:11.016221 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.019977 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.023680 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0307 19:36:11.023815 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0307 19:36:11.062030 762831 cri.go:89] found id: "ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:11.062103 762831 cri.go:89] found id: "f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:11.062141 762831 cri.go:89] found id: ""
I0307 19:36:11.062151 762831 logs.go:276] 2 containers: [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093]
I0307 19:36:11.062215 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.066359 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.070205 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0307 19:36:11.070279 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0307 19:36:11.109710 762831 cri.go:89] found id: "90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:11.109783 762831 cri.go:89] found id: ""
I0307 19:36:11.109813 762831 logs.go:276] 1 containers: [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a]
I0307 19:36:11.109911 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.114323 762831 logs.go:123] Gathering logs for kube-apiserver [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020] ...
I0307 19:36:11.114404 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:11.176440 762831 logs.go:123] Gathering logs for etcd [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d] ...
I0307 19:36:11.176475 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:11.230993 762831 logs.go:123] Gathering logs for kube-scheduler [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f] ...
I0307 19:36:11.231024 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:11.300327 762831 logs.go:123] Gathering logs for kube-proxy [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5] ...
I0307 19:36:11.300359 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:11.354977 762831 logs.go:123] Gathering logs for storage-provisioner [f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093] ...
I0307 19:36:11.355007 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:11.398803 762831 logs.go:123] Gathering logs for containerd ...
I0307 19:36:11.398834 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0307 19:36:11.458507 762831 logs.go:123] Gathering logs for kube-controller-manager [f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08] ...
I0307 19:36:11.458543 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:11.532204 762831 logs.go:123] Gathering logs for kubernetes-dashboard [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a] ...
I0307 19:36:11.535311 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:11.601998 762831 logs.go:123] Gathering logs for describe nodes ...
I0307 19:36:11.602025 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0307 19:36:11.766962 762831 logs.go:123] Gathering logs for kube-apiserver [99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939] ...
I0307 19:36:11.766998 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:11.869356 762831 logs.go:123] Gathering logs for coredns [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b] ...
I0307 19:36:11.869391 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:11.909582 762831 logs.go:123] Gathering logs for coredns [f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384] ...
I0307 19:36:11.909612 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:11.955107 762831 logs.go:123] Gathering logs for kube-proxy [5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907] ...
I0307 19:36:11.955137 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:11.995979 762831 logs.go:123] Gathering logs for kube-controller-manager [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916] ...
I0307 19:36:11.996008 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:12.056706 762831 logs.go:123] Gathering logs for dmesg ...
I0307 19:36:12.056737 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0307 19:36:12.076984 762831 logs.go:123] Gathering logs for kindnet [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57] ...
I0307 19:36:12.077015 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:12.120010 762831 logs.go:123] Gathering logs for container status ...
I0307 19:36:12.120041 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0307 19:36:12.163174 762831 logs.go:123] Gathering logs for kubelet ...
I0307 19:36:12.163205 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0307 19:36:12.218431 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562011 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-vdtz2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vdtz2" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.218682 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562119 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.218920 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562202 662 reflector.go:138] object-"kube-system"/"coredns-token-zckzk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zckzk" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.219145 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562259 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220131 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562331 662 reflector.go:138] object-"kube-system"/"kindnet-token-g7knv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g7knv" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220392 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562675 662 reflector.go:138] object-"default"/"default-token-rgss9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-rgss9" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220637 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562749 662 reflector.go:138] object-"kube-system"/"metrics-server-token-t79rr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-t79rr" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220886 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.672443 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k4nsq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k4nsq" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.233667 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.291729 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.233913 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.749946 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.236687 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:07 old-k8s-version-490121 kubelet[662]: E0307 19:31:07.413698 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.238386 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:20 old-k8s-version-490121 kubelet[662]: E0307 19:31:20.430619 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.239318 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:23 old-k8s-version-490121 kubelet[662]: E0307 19:31:23.898812 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.239668 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:24 old-k8s-version-490121 kubelet[662]: E0307 19:31:24.894861 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.240124 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:25 old-k8s-version-490121 kubelet[662]: E0307 19:31:25.899230 662 pod_workers.go:191] Error syncing pod 57707985-e3ad-463c-a7a9-150bfe271af7 ("storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"
W0307 19:36:12.242571 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:31 old-k8s-version-490121 kubelet[662]: E0307 19:31:31.411639 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.242927 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:32 old-k8s-version-490121 kubelet[662]: E0307 19:31:32.839916 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.243723 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.406752 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.244204 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.954053 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.244574 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:52 old-k8s-version-490121 kubelet[662]: E0307 19:31:52.840518 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.244771 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:57 old-k8s-version-490121 kubelet[662]: E0307 19:31:57.404922 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.245369 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.021627 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.249632 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.404313 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.249991 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:12 old-k8s-version-490121 kubelet[662]: E0307 19:32:12.840494 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.252464 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:19 old-k8s-version-490121 kubelet[662]: E0307 19:32:19.412896 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.252795 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:25 old-k8s-version-490121 kubelet[662]: E0307 19:32:25.404097 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.252980 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:34 old-k8s-version-490121 kubelet[662]: E0307 19:32:34.404539 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.253309 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:38 old-k8s-version-490121 kubelet[662]: E0307 19:32:38.404439 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.253495 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:49 old-k8s-version-490121 kubelet[662]: E0307 19:32:49.404401 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.254222 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:51 old-k8s-version-490121 kubelet[662]: E0307 19:32:51.130308 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.254581 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:52 old-k8s-version-490121 kubelet[662]: E0307 19:32:52.840092 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.254790 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:02 old-k8s-version-490121 kubelet[662]: E0307 19:33:02.404637 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.255142 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:05 old-k8s-version-490121 kubelet[662]: E0307 19:33:05.404490 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.255352 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:14 old-k8s-version-490121 kubelet[662]: E0307 19:33:14.404856 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.255709 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:17 old-k8s-version-490121 kubelet[662]: E0307 19:33:17.403976 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.255916 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:27 old-k8s-version-490121 kubelet[662]: E0307 19:33:27.404480 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.256274 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:30 old-k8s-version-490121 kubelet[662]: E0307 19:33:30.404449 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.256625 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:41 old-k8s-version-490121 kubelet[662]: E0307 19:33:41.404057 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.260957 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:42 old-k8s-version-490121 kubelet[662]: E0307 19:33:42.415836 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.261998 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:52 old-k8s-version-490121 kubelet[662]: E0307 19:33:52.404978 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.262221 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:55 old-k8s-version-490121 kubelet[662]: E0307 19:33:55.404850 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.264387 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:07 old-k8s-version-490121 kubelet[662]: E0307 19:34:07.404060 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.264606 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:09 old-k8s-version-490121 kubelet[662]: E0307 19:34:09.404722 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.265123 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:22 old-k8s-version-490121 kubelet[662]: E0307 19:34:22.325297 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.266214 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.328654 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.266457 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.404300 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.266819 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:34 old-k8s-version-490121 kubelet[662]: E0307 19:34:34.405645 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.267028 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:35 old-k8s-version-490121 kubelet[662]: E0307 19:34:35.404699 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.267382 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:47 old-k8s-version-490121 kubelet[662]: E0307 19:34:47.404007 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.267620 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:49 old-k8s-version-490121 kubelet[662]: E0307 19:34:49.404434 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.267992 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:59 old-k8s-version-490121 kubelet[662]: E0307 19:34:59.404258 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.268205 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:02 old-k8s-version-490121 kubelet[662]: E0307 19:35:02.405853 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.268570 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: E0307 19:35:12.405265 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.268783 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:17 old-k8s-version-490121 kubelet[662]: E0307 19:35:17.404385 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.269134 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: E0307 19:35:27.406835 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.269703 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:32 old-k8s-version-490121 kubelet[662]: E0307 19:35:32.407052 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.270066 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: E0307 19:35:40.404052 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.270279 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.270632 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.270841 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.271203 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.271558 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:12.271580 762831 logs.go:123] Gathering logs for etcd [05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190] ...
I0307 19:36:12.271605 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:12.320083 762831 logs.go:123] Gathering logs for kube-scheduler [156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4] ...
I0307 19:36:12.320115 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:12.372179 762831 logs.go:123] Gathering logs for kindnet [1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29] ...
I0307 19:36:12.372386 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:12.424384 762831 logs.go:123] Gathering logs for storage-provisioner [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa] ...
I0307 19:36:12.424413 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:12.473228 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:12.473254 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0307 19:36:12.473310 762831 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0307 19:36:12.473329 762831 out.go:239] Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.473339 762831 out.go:239] Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.473348 762831 out.go:239] Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.473363 762831 out.go:239] Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.473369 762831 out.go:239] Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:12.473377 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:12.473382 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:36:22.474361 762831 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0307 19:36:22.494422 762831 api_server.go:72] duration metric: took 5m48.072661998s to wait for apiserver process to appear ...
I0307 19:36:22.494445 762831 api_server.go:88] waiting for apiserver healthz status ...
I0307 19:36:22.494486 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0307 19:36:22.494542 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0307 19:36:22.559595 762831 cri.go:89] found id: "ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:22.559616 762831 cri.go:89] found id: "99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:22.559624 762831 cri.go:89] found id: ""
I0307 19:36:22.559632 762831 logs.go:276] 2 containers: [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939]
I0307 19:36:22.559686 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.563850 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.568388 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0307 19:36:22.568464 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0307 19:36:22.628461 762831 cri.go:89] found id: "5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:22.628491 762831 cri.go:89] found id: "05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:22.628496 762831 cri.go:89] found id: ""
I0307 19:36:22.628504 762831 logs.go:276] 2 containers: [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190]
I0307 19:36:22.628558 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.632518 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.636566 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0307 19:36:22.636648 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0307 19:36:22.716216 762831 cri.go:89] found id: "9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:22.716236 762831 cri.go:89] found id: "f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:22.716241 762831 cri.go:89] found id: ""
I0307 19:36:22.716248 762831 logs.go:276] 2 containers: [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384]
I0307 19:36:22.716303 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.720114 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.726286 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0307 19:36:22.726389 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0307 19:36:22.775384 762831 cri.go:89] found id: "62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:22.775417 762831 cri.go:89] found id: "156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:22.775423 762831 cri.go:89] found id: ""
I0307 19:36:22.775431 762831 logs.go:276] 2 containers: [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4]
I0307 19:36:22.775535 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.779574 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.783105 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0307 19:36:22.783214 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0307 19:36:22.834745 762831 cri.go:89] found id: "eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:22.834766 762831 cri.go:89] found id: "5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:22.834771 762831 cri.go:89] found id: ""
I0307 19:36:22.834778 762831 logs.go:276] 2 containers: [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907]
I0307 19:36:22.834885 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.838940 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.842643 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0307 19:36:22.842766 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0307 19:36:22.885986 762831 cri.go:89] found id: "2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:22.886051 762831 cri.go:89] found id: "f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:22.886073 762831 cri.go:89] found id: ""
I0307 19:36:22.886087 762831 logs.go:276] 2 containers: [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08]
I0307 19:36:22.886147 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.889768 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.893174 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0307 19:36:22.893250 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0307 19:36:22.932715 762831 cri.go:89] found id: "18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:22.932734 762831 cri.go:89] found id: "1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:22.932739 762831 cri.go:89] found id: ""
I0307 19:36:22.932746 762831 logs.go:276] 2 containers: [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29]
I0307 19:36:22.932801 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.936637 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.940288 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0307 19:36:22.940389 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0307 19:36:22.982459 762831 cri.go:89] found id: "ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:22.982483 762831 cri.go:89] found id: "f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:22.982488 762831 cri.go:89] found id: ""
I0307 19:36:22.982495 762831 logs.go:276] 2 containers: [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093]
I0307 19:36:22.982568 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.986251 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.989380 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0307 19:36:22.989460 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0307 19:36:23.030709 762831 cri.go:89] found id: "90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:23.030732 762831 cri.go:89] found id: ""
I0307 19:36:23.030739 762831 logs.go:276] 1 containers: [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a]
I0307 19:36:23.030816 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:23.034771 762831 logs.go:123] Gathering logs for coredns [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b] ...
I0307 19:36:23.034798 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:23.078994 762831 logs.go:123] Gathering logs for kube-scheduler [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f] ...
I0307 19:36:23.079022 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:23.126348 762831 logs.go:123] Gathering logs for kube-proxy [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5] ...
I0307 19:36:23.126376 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:23.172310 762831 logs.go:123] Gathering logs for storage-provisioner [f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093] ...
I0307 19:36:23.172375 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:23.212617 762831 logs.go:123] Gathering logs for containerd ...
I0307 19:36:23.212684 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0307 19:36:23.282162 762831 logs.go:123] Gathering logs for kubelet ...
I0307 19:36:23.282199 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0307 19:36:23.342202 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562011 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-vdtz2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vdtz2" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.342735 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562119 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343078 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562202 662 reflector.go:138] object-"kube-system"/"coredns-token-zckzk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zckzk" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343584 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562259 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343891 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562331 662 reflector.go:138] object-"kube-system"/"kindnet-token-g7knv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g7knv" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.344406 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562675 662 reflector.go:138] object-"default"/"default-token-rgss9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-rgss9" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.344748 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562749 662 reflector.go:138] object-"kube-system"/"metrics-server-token-t79rr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-t79rr" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.345285 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.672443 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k4nsq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k4nsq" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.353834 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.291729 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.354037 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.749946 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.356811 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:07 old-k8s-version-490121 kubelet[662]: E0307 19:31:07.413698 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.358512 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:20 old-k8s-version-490121 kubelet[662]: E0307 19:31:20.430619 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.359433 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:23 old-k8s-version-490121 kubelet[662]: E0307 19:31:23.898812 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.359765 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:24 old-k8s-version-490121 kubelet[662]: E0307 19:31:24.894861 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.360200 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:25 old-k8s-version-490121 kubelet[662]: E0307 19:31:25.899230 662 pod_workers.go:191] Error syncing pod 57707985-e3ad-463c-a7a9-150bfe271af7 ("storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"
W0307 19:36:23.364160 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:31 old-k8s-version-490121 kubelet[662]: E0307 19:31:31.411639 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.364506 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:32 old-k8s-version-490121 kubelet[662]: E0307 19:31:32.839916 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.365283 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.406752 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.365814 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.954053 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.366145 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:52 old-k8s-version-490121 kubelet[662]: E0307 19:31:52.840518 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.366334 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:57 old-k8s-version-490121 kubelet[662]: E0307 19:31:57.404922 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.366928 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.021627 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.367115 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.404313 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.367441 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:12 old-k8s-version-490121 kubelet[662]: E0307 19:32:12.840494 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.369872 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:19 old-k8s-version-490121 kubelet[662]: E0307 19:32:19.412896 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.370699 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:25 old-k8s-version-490121 kubelet[662]: E0307 19:32:25.404097 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.370901 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:34 old-k8s-version-490121 kubelet[662]: E0307 19:32:34.404539 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.371238 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:38 old-k8s-version-490121 kubelet[662]: E0307 19:32:38.404439 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.371423 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:49 old-k8s-version-490121 kubelet[662]: E0307 19:32:49.404401 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.372010 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:51 old-k8s-version-490121 kubelet[662]: E0307 19:32:51.130308 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.372334 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:52 old-k8s-version-490121 kubelet[662]: E0307 19:32:52.840092 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.372514 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:02 old-k8s-version-490121 kubelet[662]: E0307 19:33:02.404637 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.372838 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:05 old-k8s-version-490121 kubelet[662]: E0307 19:33:05.404490 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.373020 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:14 old-k8s-version-490121 kubelet[662]: E0307 19:33:14.404856 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.373343 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:17 old-k8s-version-490121 kubelet[662]: E0307 19:33:17.403976 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.373536 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:27 old-k8s-version-490121 kubelet[662]: E0307 19:33:27.404480 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.373887 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:30 old-k8s-version-490121 kubelet[662]: E0307 19:33:30.404449 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.374213 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:41 old-k8s-version-490121 kubelet[662]: E0307 19:33:41.404057 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.376719 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:42 old-k8s-version-490121 kubelet[662]: E0307 19:33:42.415836 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.377054 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:52 old-k8s-version-490121 kubelet[662]: E0307 19:33:52.404978 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.377238 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:55 old-k8s-version-490121 kubelet[662]: E0307 19:33:55.404850 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.377572 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:07 old-k8s-version-490121 kubelet[662]: E0307 19:34:07.404060 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.377754 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:09 old-k8s-version-490121 kubelet[662]: E0307 19:34:09.404722 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.378206 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:22 old-k8s-version-490121 kubelet[662]: E0307 19:34:22.325297 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.378658 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.328654 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.378843 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.404300 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.379171 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:34 old-k8s-version-490121 kubelet[662]: E0307 19:34:34.405645 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.379355 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:35 old-k8s-version-490121 kubelet[662]: E0307 19:34:35.404699 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.379679 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:47 old-k8s-version-490121 kubelet[662]: E0307 19:34:47.404007 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.379860 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:49 old-k8s-version-490121 kubelet[662]: E0307 19:34:49.404434 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.380186 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:59 old-k8s-version-490121 kubelet[662]: E0307 19:34:59.404258 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.380367 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:02 old-k8s-version-490121 kubelet[662]: E0307 19:35:02.405853 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.380716 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: E0307 19:35:12.405265 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.380901 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:17 old-k8s-version-490121 kubelet[662]: E0307 19:35:17.404385 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.381224 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: E0307 19:35:27.406835 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.381405 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:32 old-k8s-version-490121 kubelet[662]: E0307 19:35:32.407052 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.381735 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: E0307 19:35:40.404052 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.382360 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.382702 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.382887 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.383217 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.383399 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.383723 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.383904 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:23.383915 762831 logs.go:123] Gathering logs for kube-apiserver [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020] ...
I0307 19:36:23.383933 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:23.476177 762831 logs.go:123] Gathering logs for kube-controller-manager [f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08] ...
I0307 19:36:23.476227 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:23.572070 762831 logs.go:123] Gathering logs for storage-provisioner [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa] ...
I0307 19:36:23.572107 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:23.617823 762831 logs.go:123] Gathering logs for kubernetes-dashboard [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a] ...
I0307 19:36:23.617853 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:23.696040 762831 logs.go:123] Gathering logs for container status ...
I0307 19:36:23.696108 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0307 19:36:23.762456 762831 logs.go:123] Gathering logs for kube-apiserver [99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939] ...
I0307 19:36:23.762532 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:23.835674 762831 logs.go:123] Gathering logs for etcd [05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190] ...
I0307 19:36:23.835750 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:23.903568 762831 logs.go:123] Gathering logs for coredns [f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384] ...
I0307 19:36:23.903699 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:23.952099 762831 logs.go:123] Gathering logs for kindnet [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57] ...
I0307 19:36:23.952172 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:24.014266 762831 logs.go:123] Gathering logs for kindnet [1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29] ...
I0307 19:36:24.014344 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:24.149466 762831 logs.go:123] Gathering logs for describe nodes ...
I0307 19:36:24.149546 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0307 19:36:24.364732 762831 logs.go:123] Gathering logs for etcd [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d] ...
I0307 19:36:24.364896 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:24.422381 762831 logs.go:123] Gathering logs for kube-proxy [5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907] ...
I0307 19:36:24.422452 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:24.472397 762831 logs.go:123] Gathering logs for kube-controller-manager [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916] ...
I0307 19:36:24.472461 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:24.554174 762831 logs.go:123] Gathering logs for dmesg ...
I0307 19:36:24.554248 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0307 19:36:24.577804 762831 logs.go:123] Gathering logs for kube-scheduler [156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4] ...
I0307 19:36:24.577881 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:24.642561 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:24.642626 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0307 19:36:24.642701 762831 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0307 19:36:24.642745 762831 out.go:239] Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:24.642899 762831 out.go:239] Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:24.642937 762831 out.go:239] Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:24.643025 762831 out.go:239] Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:24.643060 762831 out.go:239] Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:24.643094 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:24.643117 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:36:34.644283 762831 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0307 19:36:34.654169 762831 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0307 19:36:34.656597 762831 out.go:177]
W0307 19:36:34.658535 762831 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0307 19:36:34.658604 762831 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0307 19:36:34.658626 762831 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0307 19:36:34.658632 762831 out.go:239] *
*
W0307 19:36:34.659694 762831 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0307 19:36:34.662993 762831 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-490121 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-490121
helpers_test.go:235: (dbg) docker inspect old-k8s-version-490121:
-- stdout --
[
{
"Id": "88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0",
"Created": "2024-03-07T19:27:15.489759719Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 763020,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-03-07T19:30:26.830428213Z",
"FinishedAt": "2024-03-07T19:30:25.780938358Z"
},
"Image": "sha256:4a9b65157dd7fb2ddb7cb7afe975b3dc288e9877c60d13613a69dd41a70e2e4e",
"ResolvConfPath": "/var/lib/docker/containers/88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0/hostname",
"HostsPath": "/var/lib/docker/containers/88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0/hosts",
"LogPath": "/var/lib/docker/containers/88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0/88e4e15b19a2d24455aa9f8d367b5d28f54f7e1bcdb4dfde532231a256de22c0-json.log",
"Name": "/old-k8s-version-490121",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-490121:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-490121",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/abbc2bc05533b6ec1f7375adc27f3375613677eb4ca0d72ab824571d558d2fad-init/diff:/var/lib/docker/overlay2/0f2c2bc9ebcb6a090c4ed5f3df98eb2fb852fa3a78be98cc34cd75b1870e6d76/diff",
"MergedDir": "/var/lib/docker/overlay2/abbc2bc05533b6ec1f7375adc27f3375613677eb4ca0d72ab824571d558d2fad/merged",
"UpperDir": "/var/lib/docker/overlay2/abbc2bc05533b6ec1f7375adc27f3375613677eb4ca0d72ab824571d558d2fad/diff",
"WorkDir": "/var/lib/docker/overlay2/abbc2bc05533b6ec1f7375adc27f3375613677eb4ca0d72ab824571d558d2fad/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-490121",
"Source": "/var/lib/docker/volumes/old-k8s-version-490121/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-490121",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-490121",
"name.minikube.sigs.k8s.io": "old-k8s-version-490121",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "48c445440d40d4683a57daf47cc0316fa469d8436433a390600e88774af70963",
"SandboxKey": "/var/run/docker/netns/48c445440d40",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33808"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33807"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33804"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33806"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33805"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-490121": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": [
"88e4e15b19a2",
"old-k8s-version-490121"
],
"MacAddress": "02:42:c0:a8:55:02",
"NetworkID": "83c5f395470026aef4ea22efddb9b88956697e7a6581a4d5a36fac9ca0f95270",
"EndpointID": "0c6f32f6f314215ee283975bb256db2fd6c1a8398e27a9dd18effa68d75b764f",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-490121",
"88e4e15b19a2"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-490121 -n old-k8s-version-490121
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-490121 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-490121 logs -n 25: (2.079197574s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| start | -p force-systemd-flag-093673 | force-systemd-flag-093673 | jenkins | v1.32.0 | 07 Mar 24 19:25 UTC | 07 Mar 24 19:26 UTC |
| | --memory=2048 --force-systemd | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-flag-093673 | force-systemd-flag-093673 | jenkins | v1.32.0 | 07 Mar 24 19:26 UTC | 07 Mar 24 19:26 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-flag-093673 | force-systemd-flag-093673 | jenkins | v1.32.0 | 07 Mar 24 19:26 UTC | 07 Mar 24 19:26 UTC |
| start | -p cert-options-533978 | cert-options-533978 | jenkins | v1.32.0 | 07 Mar 24 19:26 UTC | 07 Mar 24 19:27 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-533978 ssh | cert-options-533978 | jenkins | v1.32.0 | 07 Mar 24 19:27 UTC | 07 Mar 24 19:27 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-533978 -- sudo | cert-options-533978 | jenkins | v1.32.0 | 07 Mar 24 19:27 UTC | 07 Mar 24 19:27 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-533978 | cert-options-533978 | jenkins | v1.32.0 | 07 Mar 24 19:27 UTC | 07 Mar 24 19:27 UTC |
| start | -p old-k8s-version-490121 | old-k8s-version-490121 | jenkins | v1.32.0 | 07 Mar 24 19:27 UTC | 07 Mar 24 19:30 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-643500 | cert-expiration-643500 | jenkins | v1.32.0 | 07 Mar 24 19:29 UTC | 07 Mar 24 19:29 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-643500 | cert-expiration-643500 | jenkins | v1.32.0 | 07 Mar 24 19:29 UTC | 07 Mar 24 19:29 UTC |
| start | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:29 UTC | 07 Mar 24 19:30 UTC |
| | --memory=2200 --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.29.0-rc.2 | | | | | |
| addons | enable metrics-server -p old-k8s-version-490121 | old-k8s-version-490121 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-490121 | old-k8s-version-490121 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-490121 | old-k8s-version-490121 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-490121 | old-k8s-version-490121 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:30 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:30 UTC | 07 Mar 24 19:35 UTC |
| | --memory=2200 --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.29.0-rc.2 | | | | | |
| image | no-preload-028045 image list | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | 07 Mar 24 19:35 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | 07 Mar 24 19:35 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | 07 Mar 24 19:35 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | 07 Mar 24 19:35 UTC |
| delete | -p no-preload-028045 | no-preload-028045 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | 07 Mar 24 19:35 UTC |
| start | -p embed-certs-327564 | embed-certs-327564 | jenkins | v1.32.0 | 07 Mar 24 19:35 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.28.4 | | | | | |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/03/07 19:35:31
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.22.0 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0307 19:35:31.855739 771734 out.go:291] Setting OutFile to fd 1 ...
I0307 19:35:31.855918 771734 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:35:31.855946 771734 out.go:304] Setting ErrFile to fd 2...
I0307 19:35:31.855970 771734 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:35:31.856240 771734 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18239-558171/.minikube/bin
I0307 19:35:31.856717 771734 out.go:298] Setting JSON to false
I0307 19:35:31.857832 771734 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":11876,"bootTime":1709828256,"procs":221,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0307 19:35:31.857934 771734 start.go:139] virtualization:
I0307 19:35:31.860533 771734 out.go:177] * [embed-certs-327564] minikube v1.32.0 on Ubuntu 20.04 (arm64)
I0307 19:35:31.863336 771734 out.go:177] - MINIKUBE_LOCATION=18239
I0307 19:35:31.865010 771734 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0307 19:35:31.863470 771734 notify.go:220] Checking for updates...
I0307 19:35:31.867149 771734 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18239-558171/kubeconfig
I0307 19:35:31.869263 771734 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18239-558171/.minikube
I0307 19:35:31.870993 771734 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0307 19:35:31.873118 771734 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0307 19:35:31.875699 771734 config.go:182] Loaded profile config "old-k8s-version-490121": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0307 19:35:31.875866 771734 driver.go:392] Setting default libvirt URI to qemu:///system
I0307 19:35:31.897777 771734 docker.go:122] docker version: linux-25.0.4:Docker Engine - Community
I0307 19:35:31.897902 771734 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0307 19:35:31.963800 771734 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-07 19:35:31.953865224 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0307 19:35:31.963914 771734 docker.go:295] overlay module found
I0307 19:35:31.966513 771734 out.go:177] * Using the docker driver based on user configuration
I0307 19:35:31.968866 771734 start.go:297] selected driver: docker
I0307 19:35:31.968884 771734 start.go:901] validating driver "docker" against <nil>
I0307 19:35:31.968904 771734 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0307 19:35:31.969693 771734 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0307 19:35:32.031093 771734 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-07 19:35:32.021090322 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0307 19:35:32.031266 771734 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0307 19:35:32.031498 771734 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0307 19:35:32.033678 771734 out.go:177] * Using Docker driver with root privileges
I0307 19:35:32.035709 771734 cni.go:84] Creating CNI manager for ""
I0307 19:35:32.035733 771734 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0307 19:35:32.035743 771734 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0307 19:35:32.035847 771734 start.go:340] cluster config:
{Name:embed-certs-327564 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-327564 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stati
cIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0307 19:35:32.040597 771734 out.go:177] * Starting "embed-certs-327564" primary control-plane node in "embed-certs-327564" cluster
I0307 19:35:32.043149 771734 cache.go:121] Beginning downloading kic base image for docker with containerd
I0307 19:35:32.045428 771734 out.go:177] * Pulling base image v0.0.42-1708944392-18244 ...
I0307 19:35:32.047530 771734 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon
I0307 19:35:32.047606 771734 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0307 19:35:32.047643 771734 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4
I0307 19:35:32.047655 771734 cache.go:56] Caching tarball of preloaded images
I0307 19:35:32.047750 771734 preload.go:173] Found /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0307 19:35:32.047765 771734 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on containerd
I0307 19:35:32.047869 771734 profile.go:142] Saving config to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/config.json ...
I0307 19:35:32.047890 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/config.json: {Name:mkbcd98b08a982b8279658fceb8e97a6b892072f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:32.064555 771734 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon, skipping pull
I0307 19:35:32.064586 771734 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in daemon, skipping load
I0307 19:35:32.064607 771734 cache.go:194] Successfully downloaded all kic artifacts
I0307 19:35:32.064636 771734 start.go:360] acquireMachinesLock for embed-certs-327564: {Name:mk4731e9259edf2aeab2afb8a48ec2b1d2c396bf Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0307 19:35:32.065206 771734 start.go:364] duration metric: took 545.819µs to acquireMachinesLock for "embed-certs-327564"
I0307 19:35:32.065246 771734 start.go:93] Provisioning new machine with config: &{Name:embed-certs-327564 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-327564 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableM
etrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0307 19:35:32.065345 771734 start.go:125] createHost starting for "" (driver="docker")
I0307 19:35:31.606730 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:33.656753 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:36.129051 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:32.068491 771734 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0307 19:35:32.068751 771734 start.go:159] libmachine.API.Create for "embed-certs-327564" (driver="docker")
I0307 19:35:32.068789 771734 client.go:168] LocalClient.Create starting
I0307 19:35:32.068873 771734 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem
I0307 19:35:32.068912 771734 main.go:141] libmachine: Decoding PEM data...
I0307 19:35:32.068930 771734 main.go:141] libmachine: Parsing certificate...
I0307 19:35:32.068987 771734 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem
I0307 19:35:32.069017 771734 main.go:141] libmachine: Decoding PEM data...
I0307 19:35:32.069027 771734 main.go:141] libmachine: Parsing certificate...
I0307 19:35:32.069408 771734 cli_runner.go:164] Run: docker network inspect embed-certs-327564 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0307 19:35:32.085099 771734 cli_runner.go:211] docker network inspect embed-certs-327564 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0307 19:35:32.085192 771734 network_create.go:281] running [docker network inspect embed-certs-327564] to gather additional debugging logs...
I0307 19:35:32.085214 771734 cli_runner.go:164] Run: docker network inspect embed-certs-327564
W0307 19:35:32.100716 771734 cli_runner.go:211] docker network inspect embed-certs-327564 returned with exit code 1
I0307 19:35:32.100748 771734 network_create.go:284] error running [docker network inspect embed-certs-327564]: docker network inspect embed-certs-327564: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-327564 not found
I0307 19:35:32.100760 771734 network_create.go:286] output of [docker network inspect embed-certs-327564]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-327564 not found
** /stderr **
I0307 19:35:32.100880 771734 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0307 19:35:32.122953 771734 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-182dc02cf5ca IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:0e:2f:64:c2} reservation:<nil>}
I0307 19:35:32.123432 771734 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-f2f61e5e37e0 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:ec:4e:01:50} reservation:<nil>}
I0307 19:35:32.123887 771734 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-3ac39cac33a3 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:37:c5:77:12} reservation:<nil>}
I0307 19:35:32.124432 771734 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x40025b9180}
I0307 19:35:32.124461 771734 network_create.go:124] attempt to create docker network embed-certs-327564 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0307 19:35:32.124520 771734 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-327564 embed-certs-327564
I0307 19:35:32.186227 771734 network_create.go:108] docker network embed-certs-327564 192.168.76.0/24 created
I0307 19:35:32.186268 771734 kic.go:121] calculated static IP "192.168.76.2" for the "embed-certs-327564" container
I0307 19:35:32.186363 771734 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0307 19:35:32.202409 771734 cli_runner.go:164] Run: docker volume create embed-certs-327564 --label name.minikube.sigs.k8s.io=embed-certs-327564 --label created_by.minikube.sigs.k8s.io=true
I0307 19:35:32.218608 771734 oci.go:103] Successfully created a docker volume embed-certs-327564
I0307 19:35:32.218688 771734 cli_runner.go:164] Run: docker run --rm --name embed-certs-327564-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-327564 --entrypoint /usr/bin/test -v embed-certs-327564:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -d /var/lib
I0307 19:35:32.911513 771734 oci.go:107] Successfully prepared a docker volume embed-certs-327564
I0307 19:35:32.911596 771734 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0307 19:35:32.911616 771734 kic.go:194] Starting extracting preloaded images to volume ...
I0307 19:35:32.911716 771734 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-327564:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir
I0307 19:35:38.606994 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:41.105989 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:38.541711 771734 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18239-558171/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-327564:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir: (5.629954813s)
I0307 19:35:38.541743 771734 kic.go:203] duration metric: took 5.63012328s to extract preloaded images to volume ...
W0307 19:35:38.541891 771734 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0307 19:35:38.542003 771734 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0307 19:35:38.596251 771734 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-327564 --name embed-certs-327564 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-327564 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-327564 --network embed-certs-327564 --ip 192.168.76.2 --volume embed-certs-327564:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08
I0307 19:35:38.931797 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Running}}
I0307 19:35:38.965168 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:35:38.985759 771734 cli_runner.go:164] Run: docker exec embed-certs-327564 stat /var/lib/dpkg/alternatives/iptables
I0307 19:35:39.055098 771734 oci.go:144] the created container "embed-certs-327564" has a running status.
I0307 19:35:39.055136 771734 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa...
I0307 19:35:39.459163 771734 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0307 19:35:39.488783 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:35:39.521171 771734 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0307 19:35:39.521190 771734 kic_runner.go:114] Args: [docker exec --privileged embed-certs-327564 chown docker:docker /home/docker/.ssh/authorized_keys]
I0307 19:35:39.600041 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:35:39.628432 771734 machine.go:94] provisionDockerMachine start ...
I0307 19:35:39.628518 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:39.657856 771734 main.go:141] libmachine: Using SSH client type: native
I0307 19:35:39.658142 771734 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33818 <nil> <nil>}
I0307 19:35:39.658158 771734 main.go:141] libmachine: About to run SSH command:
hostname
I0307 19:35:39.658955 771734 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:59958->127.0.0.1:33818: read: connection reset by peer
I0307 19:35:42.792994 771734 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-327564
I0307 19:35:42.793020 771734 ubuntu.go:169] provisioning hostname "embed-certs-327564"
I0307 19:35:42.793096 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:42.812606 771734 main.go:141] libmachine: Using SSH client type: native
I0307 19:35:42.812858 771734 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33818 <nil> <nil>}
I0307 19:35:42.812884 771734 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-327564 && echo "embed-certs-327564" | sudo tee /etc/hostname
I0307 19:35:42.958405 771734 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-327564
I0307 19:35:42.958483 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:42.976423 771734 main.go:141] libmachine: Using SSH client type: native
I0307 19:35:42.976669 771734 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil> [] 0s} 127.0.0.1 33818 <nil> <nil>}
I0307 19:35:42.976686 771734 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-327564' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-327564/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-327564' | sudo tee -a /etc/hosts;
fi
fi
I0307 19:35:43.108420 771734 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0307 19:35:43.108502 771734 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18239-558171/.minikube CaCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18239-558171/.minikube}
I0307 19:35:43.108561 771734 ubuntu.go:177] setting up certificates
I0307 19:35:43.108590 771734 provision.go:84] configureAuth start
I0307 19:35:43.108705 771734 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-327564
I0307 19:35:43.124220 771734 provision.go:143] copyHostCerts
I0307 19:35:43.124285 771734 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem, removing ...
I0307 19:35:43.124294 771734 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem
I0307 19:35:43.124376 771734 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/ca.pem (1082 bytes)
I0307 19:35:43.124462 771734 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem, removing ...
I0307 19:35:43.124467 771734 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem
I0307 19:35:43.124497 771734 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/cert.pem (1123 bytes)
I0307 19:35:43.124546 771734 exec_runner.go:144] found /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem, removing ...
I0307 19:35:43.124551 771734 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem
I0307 19:35:43.124572 771734 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18239-558171/.minikube/key.pem (1675 bytes)
I0307 19:35:43.124616 771734 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem org=jenkins.embed-certs-327564 san=[127.0.0.1 192.168.76.2 embed-certs-327564 localhost minikube]
I0307 19:35:43.278562 771734 provision.go:177] copyRemoteCerts
I0307 19:35:43.278666 771734 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0307 19:35:43.278738 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:43.294583 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:35:43.390579 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0307 19:35:43.416544 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0307 19:35:43.441719 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0307 19:35:43.467195 771734 provision.go:87] duration metric: took 358.566729ms to configureAuth
I0307 19:35:43.467224 771734 ubuntu.go:193] setting minikube options for container-runtime
I0307 19:35:43.467410 771734 config.go:182] Loaded profile config "embed-certs-327564": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.28.4
I0307 19:35:43.467424 771734 machine.go:97] duration metric: took 3.838975151s to provisionDockerMachine
I0307 19:35:43.467431 771734 client.go:171] duration metric: took 11.398632348s to LocalClient.Create
I0307 19:35:43.467445 771734 start.go:167] duration metric: took 11.398696496s to libmachine.API.Create "embed-certs-327564"
I0307 19:35:43.467452 771734 start.go:293] postStartSetup for "embed-certs-327564" (driver="docker")
I0307 19:35:43.467462 771734 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0307 19:35:43.467518 771734 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0307 19:35:43.467573 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:43.483356 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:35:43.582641 771734 ssh_runner.go:195] Run: cat /etc/os-release
I0307 19:35:43.586218 771734 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0307 19:35:43.586257 771734 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0307 19:35:43.586268 771734 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0307 19:35:43.586275 771734 info.go:137] Remote host: Ubuntu 22.04.3 LTS
I0307 19:35:43.586286 771734 filesync.go:126] Scanning /home/jenkins/minikube-integration/18239-558171/.minikube/addons for local assets ...
I0307 19:35:43.586351 771734 filesync.go:126] Scanning /home/jenkins/minikube-integration/18239-558171/.minikube/files for local assets ...
I0307 19:35:43.586435 771734 filesync.go:149] local asset: /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem -> 5635812.pem in /etc/ssl/certs
I0307 19:35:43.586553 771734 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0307 19:35:43.595355 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem --> /etc/ssl/certs/5635812.pem (1708 bytes)
I0307 19:35:43.625672 771734 start.go:296] duration metric: took 158.205709ms for postStartSetup
I0307 19:35:43.626063 771734 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-327564
I0307 19:35:43.641961 771734 profile.go:142] Saving config to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/config.json ...
I0307 19:35:43.642249 771734 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0307 19:35:43.642319 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:43.674467 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:35:43.766317 771734 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0307 19:35:43.771235 771734 start.go:128] duration metric: took 11.705875012s to createHost
I0307 19:35:43.771257 771734 start.go:83] releasing machines lock for "embed-certs-327564", held for 11.706031745s
I0307 19:35:43.771331 771734 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-327564
I0307 19:35:43.787492 771734 ssh_runner.go:195] Run: cat /version.json
I0307 19:35:43.787543 771734 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0307 19:35:43.787553 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:43.787606 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:35:43.817766 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:35:43.827216 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:35:43.916988 771734 ssh_runner.go:195] Run: systemctl --version
I0307 19:35:44.036940 771734 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0307 19:35:44.041298 771734 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0307 19:35:44.069506 771734 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0307 19:35:44.069710 771734 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0307 19:35:44.101236 771734 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0307 19:35:44.101261 771734 start.go:494] detecting cgroup driver to use...
I0307 19:35:44.101294 771734 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0307 19:35:44.101347 771734 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0307 19:35:44.117392 771734 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0307 19:35:44.130342 771734 docker.go:217] disabling cri-docker service (if available) ...
I0307 19:35:44.130425 771734 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0307 19:35:44.144765 771734 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0307 19:35:44.159739 771734 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0307 19:35:44.254737 771734 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0307 19:35:44.361182 771734 docker.go:233] disabling docker service ...
I0307 19:35:44.361266 771734 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0307 19:35:44.387621 771734 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0307 19:35:44.410439 771734 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0307 19:35:44.530474 771734 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0307 19:35:44.627638 771734 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0307 19:35:44.639002 771734 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0307 19:35:44.655805 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I0307 19:35:44.666038 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0307 19:35:44.676278 771734 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0307 19:35:44.676356 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0307 19:35:44.687391 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0307 19:35:44.696975 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0307 19:35:44.708461 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0307 19:35:44.718031 771734 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0307 19:35:44.726924 771734 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0307 19:35:44.736667 771734 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0307 19:35:44.745319 771734 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0307 19:35:44.754586 771734 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:35:44.849367 771734 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0307 19:35:44.996466 771734 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0307 19:35:44.996610 771734 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0307 19:35:45.000581 771734 start.go:562] Will wait 60s for crictl version
I0307 19:35:45.000709 771734 ssh_runner.go:195] Run: which crictl
I0307 19:35:45.005814 771734 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0307 19:35:45.085227 771734 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0307 19:35:45.085390 771734 ssh_runner.go:195] Run: containerd --version
I0307 19:35:45.129159 771734 ssh_runner.go:195] Run: containerd --version
I0307 19:35:45.164974 771734 out.go:177] * Preparing Kubernetes v1.28.4 on containerd 1.6.28 ...
I0307 19:35:43.106627 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:45.115823 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:45.167376 771734 cli_runner.go:164] Run: docker network inspect embed-certs-327564 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0307 19:35:45.188092 771734 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0307 19:35:45.193236 771734 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0307 19:35:45.219469 771734 kubeadm.go:877] updating cluster {Name:embed-certs-327564 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-327564 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:
false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0307 19:35:45.219636 771734 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0307 19:35:45.219712 771734 ssh_runner.go:195] Run: sudo crictl images --output json
I0307 19:35:45.281810 771734 containerd.go:612] all images are preloaded for containerd runtime.
I0307 19:35:45.281841 771734 containerd.go:519] Images already preloaded, skipping extraction
I0307 19:35:45.281911 771734 ssh_runner.go:195] Run: sudo crictl images --output json
I0307 19:35:45.329888 771734 containerd.go:612] all images are preloaded for containerd runtime.
I0307 19:35:45.329914 771734 cache_images.go:84] Images are preloaded, skipping loading
I0307 19:35:45.329922 771734 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.28.4 containerd true true} ...
I0307 19:35:45.330073 771734 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.28.4/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-327564 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.28.4 ClusterName:embed-certs-327564 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0307 19:35:45.330166 771734 ssh_runner.go:195] Run: sudo crictl info
I0307 19:35:45.376804 771734 cni.go:84] Creating CNI manager for ""
I0307 19:35:45.376831 771734 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0307 19:35:45.376842 771734 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0307 19:35:45.376887 771734 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.28.4 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-327564 NodeName:embed-certs-327564 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0307 19:35:45.377042 771734 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-327564"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.28.4
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0307 19:35:45.377132 771734 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.4
I0307 19:35:45.386893 771734 binaries.go:44] Found k8s binaries, skipping transfer
I0307 19:35:45.387009 771734 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0307 19:35:45.396611 771734 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I0307 19:35:45.418050 771734 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0307 19:35:45.437290 771734 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I0307 19:35:45.457239 771734 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0307 19:35:45.460855 771734 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0307 19:35:45.473274 771734 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:35:45.575696 771734 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0307 19:35:45.599165 771734 certs.go:68] Setting up /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564 for IP: 192.168.76.2
I0307 19:35:45.599207 771734 certs.go:194] generating shared ca certs ...
I0307 19:35:45.599225 771734 certs.go:226] acquiring lock for ca certs: {Name:mke14792b1616e9503645c7147aed38043ea5d64 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:45.599390 771734 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18239-558171/.minikube/ca.key
I0307 19:35:45.599438 771734 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.key
I0307 19:35:45.599450 771734 certs.go:256] generating profile certs ...
I0307 19:35:45.599513 771734 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.key
I0307 19:35:45.599534 771734 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.crt with IP's: []
I0307 19:35:46.980480 771734 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.crt ...
I0307 19:35:46.980509 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.crt: {Name:mk774ef848685819579493f7e6b8079e43b50e88 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:46.981231 771734 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.key ...
I0307 19:35:46.981248 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/client.key: {Name:mk4dd5f6d1905a200873b0100af2d813ae9c8774 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:46.981702 771734 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key.2c89eb66
I0307 19:35:46.981723 771734 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt.2c89eb66 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.76.2]
I0307 19:35:47.491417 771734 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt.2c89eb66 ...
I0307 19:35:47.491453 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt.2c89eb66: {Name:mk92b131c7677506750fab443c306129e18ffab2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:47.492216 771734 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key.2c89eb66 ...
I0307 19:35:47.492244 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key.2c89eb66: {Name:mk7649ad94abeadd4e18f36496675feeb06a1f52 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:47.492917 771734 certs.go:381] copying /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt.2c89eb66 -> /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt
I0307 19:35:47.493051 771734 certs.go:385] copying /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key.2c89eb66 -> /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key
I0307 19:35:47.493125 771734 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.key
I0307 19:35:47.493152 771734 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.crt with IP's: []
I0307 19:35:48.171680 771734 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.crt ...
I0307 19:35:48.171712 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.crt: {Name:mke81cc67d6bc4c483659945a3c4feeb4c204416 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:48.171908 771734 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.key ...
I0307 19:35:48.171925 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.key: {Name:mk4d217cc90e807798441a74769a6c7aa5f0c6b5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:35:48.172118 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581.pem (1338 bytes)
W0307 19:35:48.172164 771734 certs.go:480] ignoring /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581_empty.pem, impossibly tiny 0 bytes
I0307 19:35:48.172178 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca-key.pem (1675 bytes)
I0307 19:35:48.172203 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/ca.pem (1082 bytes)
I0307 19:35:48.172232 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/cert.pem (1123 bytes)
I0307 19:35:48.172258 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/certs/key.pem (1675 bytes)
I0307 19:35:48.172304 771734 certs.go:484] found cert: /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem (1708 bytes)
I0307 19:35:48.172943 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0307 19:35:48.198072 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0307 19:35:48.222711 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0307 19:35:48.252898 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0307 19:35:48.277286 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0307 19:35:48.300988 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0307 19:35:48.325006 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0307 19:35:48.348727 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/profiles/embed-certs-327564/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0307 19:35:48.373322 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/certs/563581.pem --> /usr/share/ca-certificates/563581.pem (1338 bytes)
I0307 19:35:48.398646 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/files/etc/ssl/certs/5635812.pem --> /usr/share/ca-certificates/5635812.pem (1708 bytes)
I0307 19:35:48.425615 771734 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18239-558171/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0307 19:35:48.450725 771734 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0307 19:35:48.469827 771734 ssh_runner.go:195] Run: openssl version
I0307 19:35:48.477095 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/5635812.pem && ln -fs /usr/share/ca-certificates/5635812.pem /etc/ssl/certs/5635812.pem"
I0307 19:35:48.487390 771734 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/5635812.pem
I0307 19:35:48.490907 771734 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 7 18:50 /usr/share/ca-certificates/5635812.pem
I0307 19:35:48.490972 771734 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/5635812.pem
I0307 19:35:48.498293 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/5635812.pem /etc/ssl/certs/3ec20f2e.0"
I0307 19:35:48.507802 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0307 19:35:48.517690 771734 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0307 19:35:48.521700 771734 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 7 18:44 /usr/share/ca-certificates/minikubeCA.pem
I0307 19:35:48.521805 771734 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0307 19:35:48.528689 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0307 19:35:48.538402 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/563581.pem && ln -fs /usr/share/ca-certificates/563581.pem /etc/ssl/certs/563581.pem"
I0307 19:35:48.547792 771734 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/563581.pem
I0307 19:35:48.551414 771734 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 7 18:50 /usr/share/ca-certificates/563581.pem
I0307 19:35:48.551482 771734 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/563581.pem
I0307 19:35:48.558671 771734 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/563581.pem /etc/ssl/certs/51391683.0"
I0307 19:35:48.568475 771734 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0307 19:35:48.571930 771734 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0307 19:35:48.571981 771734 kubeadm.go:391] StartCluster: {Name:embed-certs-327564 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-327564 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fal
se CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0307 19:35:48.572065 771734 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0307 19:35:48.572122 771734 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0307 19:35:48.634324 771734 cri.go:89] found id: ""
I0307 19:35:48.634412 771734 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0307 19:35:48.649398 771734 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0307 19:35:48.660975 771734 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver
I0307 19:35:48.661148 771734 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0307 19:35:48.676050 771734 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0307 19:35:48.676130 771734 kubeadm.go:156] found existing configuration files:
I0307 19:35:48.676252 771734 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0307 19:35:48.690366 771734 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0307 19:35:48.690539 771734 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0307 19:35:48.702610 771734 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0307 19:35:48.713159 771734 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0307 19:35:48.713252 771734 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0307 19:35:48.722907 771734 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0307 19:35:48.732645 771734 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0307 19:35:48.732715 771734 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0307 19:35:48.742880 771734 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0307 19:35:48.751488 771734 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0307 19:35:48.751627 771734 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0307 19:35:48.759939 771734 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.4:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0307 19:35:48.804662 771734 kubeadm.go:309] [init] Using Kubernetes version: v1.28.4
I0307 19:35:48.805017 771734 kubeadm.go:309] [preflight] Running pre-flight checks
I0307 19:35:48.847976 771734 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:
I0307 19:35:48.848067 771734 kubeadm.go:309] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1055-aws[0m
I0307 19:35:48.848118 771734 kubeadm.go:309] [0;37mOS[0m: [0;32mLinux[0m
I0307 19:35:48.848177 771734 kubeadm.go:309] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0307 19:35:48.848235 771734 kubeadm.go:309] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0307 19:35:48.848286 771734 kubeadm.go:309] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0307 19:35:48.848345 771734 kubeadm.go:309] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0307 19:35:48.848402 771734 kubeadm.go:309] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0307 19:35:48.848462 771734 kubeadm.go:309] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0307 19:35:48.848513 771734 kubeadm.go:309] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0307 19:35:48.848582 771734 kubeadm.go:309] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0307 19:35:48.848640 771734 kubeadm.go:309] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0307 19:35:48.922663 771734 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0307 19:35:48.922848 771734 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0307 19:35:48.922998 771734 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0307 19:35:49.162474 771734 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0307 19:35:47.609214 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:50.109551 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:49.165445 771734 out.go:204] - Generating certificates and keys ...
I0307 19:35:49.165660 771734 kubeadm.go:309] [certs] Using existing ca certificate authority
I0307 19:35:49.165772 771734 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0307 19:35:50.485888 771734 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0307 19:35:50.863723 771734 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0307 19:35:51.123853 771734 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0307 19:35:51.839259 771734 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0307 19:35:52.606769 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:54.611278 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:52.579318 771734 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0307 19:35:52.579603 771734 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [embed-certs-327564 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0307 19:35:52.964269 771734 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0307 19:35:52.964582 771734 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-327564 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0307 19:35:53.372229 771734 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0307 19:35:55.022714 771734 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0307 19:35:55.341864 771734 kubeadm.go:309] [certs] Generating "sa" key and public key
I0307 19:35:55.341940 771734 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0307 19:35:55.917625 771734 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0307 19:35:56.527757 771734 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0307 19:35:56.852222 771734 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0307 19:35:57.738026 771734 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0307 19:35:57.738892 771734 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0307 19:35:57.741727 771734 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0307 19:35:57.107679 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:59.110902 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:01.134439 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:35:57.744358 771734 out.go:204] - Booting up control plane ...
I0307 19:35:57.744456 771734 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0307 19:35:57.744538 771734 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0307 19:35:57.745227 771734 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0307 19:35:57.759395 771734 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0307 19:35:57.760489 771734 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0307 19:35:57.760644 771734 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0307 19:35:57.883823 771734 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0307 19:36:03.606584 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:05.614979 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:05.887317 771734 kubeadm.go:309] [apiclient] All control plane components are healthy after 8.003640 seconds
I0307 19:36:05.887471 771734 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0307 19:36:05.902754 771734 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0307 19:36:06.427886 771734 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs
I0307 19:36:06.428085 771734 kubeadm.go:309] [mark-control-plane] Marking the node embed-certs-327564 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0307 19:36:06.940823 771734 kubeadm.go:309] [bootstrap-token] Using token: xgqx1n.1ci97fwt6benlf6i
I0307 19:36:06.942578 771734 out.go:204] - Configuring RBAC rules ...
I0307 19:36:06.942697 771734 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0307 19:36:06.951017 771734 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0307 19:36:06.962302 771734 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0307 19:36:06.967349 771734 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0307 19:36:06.972119 771734 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0307 19:36:06.977887 771734 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0307 19:36:06.993730 771734 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0307 19:36:07.297992 771734 kubeadm.go:309] [addons] Applied essential addon: CoreDNS
I0307 19:36:07.358160 771734 kubeadm.go:309] [addons] Applied essential addon: kube-proxy
I0307 19:36:07.359473 771734 kubeadm.go:309]
I0307 19:36:07.359563 771734 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!
I0307 19:36:07.359575 771734 kubeadm.go:309]
I0307 19:36:07.359659 771734 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:
I0307 19:36:07.359669 771734 kubeadm.go:309]
I0307 19:36:07.359694 771734 kubeadm.go:309] mkdir -p $HOME/.kube
I0307 19:36:07.359756 771734 kubeadm.go:309] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0307 19:36:07.359807 771734 kubeadm.go:309] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0307 19:36:07.359819 771734 kubeadm.go:309]
I0307 19:36:07.359872 771734 kubeadm.go:309] Alternatively, if you are the root user, you can run:
I0307 19:36:07.359882 771734 kubeadm.go:309]
I0307 19:36:07.359934 771734 kubeadm.go:309] export KUBECONFIG=/etc/kubernetes/admin.conf
I0307 19:36:07.359943 771734 kubeadm.go:309]
I0307 19:36:07.359996 771734 kubeadm.go:309] You should now deploy a pod network to the cluster.
I0307 19:36:07.360075 771734 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0307 19:36:07.360147 771734 kubeadm.go:309] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0307 19:36:07.360155 771734 kubeadm.go:309]
I0307 19:36:07.360236 771734 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities
I0307 19:36:07.360313 771734 kubeadm.go:309] and service account keys on each node and then running the following as root:
I0307 19:36:07.360322 771734 kubeadm.go:309]
I0307 19:36:07.360403 771734 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token xgqx1n.1ci97fwt6benlf6i \
I0307 19:36:07.360505 771734 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:17caf06007e0764138c1f585dfe115b801f228bdeee3cba3ea5bff5870a6e807 \
I0307 19:36:07.360529 771734 kubeadm.go:309] --control-plane
I0307 19:36:07.360538 771734 kubeadm.go:309]
I0307 19:36:07.360619 771734 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:
I0307 19:36:07.360628 771734 kubeadm.go:309]
I0307 19:36:07.360706 771734 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token xgqx1n.1ci97fwt6benlf6i \
I0307 19:36:07.360808 771734 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:17caf06007e0764138c1f585dfe115b801f228bdeee3cba3ea5bff5870a6e807
I0307 19:36:07.370691 771734 kubeadm.go:309] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1055-aws\n", err: exit status 1
I0307 19:36:07.370812 771734 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0307 19:36:07.370837 771734 cni.go:84] Creating CNI manager for ""
I0307 19:36:07.370849 771734 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0307 19:36:07.373311 771734 out.go:177] * Configuring CNI (Container Networking Interface) ...
I0307 19:36:08.112034 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:10.606231 762831 pod_ready.go:102] pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:10.606263 762831 pod_ready.go:81] duration metric: took 4m0.006564846s for pod "metrics-server-9975d5f86-w9hn6" in "kube-system" namespace to be "Ready" ...
E0307 19:36:10.606274 762831 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0307 19:36:10.606283 762831 pod_ready.go:38] duration metric: took 5m19.314418834s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0307 19:36:10.606296 762831 api_server.go:52] waiting for apiserver process to appear ...
I0307 19:36:10.606341 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0307 19:36:10.606408 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0307 19:36:10.656033 762831 cri.go:89] found id: "ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:10.656094 762831 cri.go:89] found id: "99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:10.656106 762831 cri.go:89] found id: ""
I0307 19:36:10.656114 762831 logs.go:276] 2 containers: [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939]
I0307 19:36:10.656170 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.659898 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.663752 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0307 19:36:10.663849 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0307 19:36:10.711143 762831 cri.go:89] found id: "5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:10.711195 762831 cri.go:89] found id: "05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:10.711214 762831 cri.go:89] found id: ""
I0307 19:36:10.711239 762831 logs.go:276] 2 containers: [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190]
I0307 19:36:10.711326 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.715686 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.719890 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0307 19:36:10.720008 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0307 19:36:10.782775 762831 cri.go:89] found id: "9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:10.782850 762831 cri.go:89] found id: "f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:10.782869 762831 cri.go:89] found id: ""
I0307 19:36:10.782892 762831 logs.go:276] 2 containers: [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384]
I0307 19:36:10.782980 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.787225 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.791833 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0307 19:36:10.791972 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0307 19:36:10.850718 762831 cri.go:89] found id: "62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:10.850794 762831 cri.go:89] found id: "156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:10.850813 762831 cri.go:89] found id: ""
I0307 19:36:10.850837 762831 logs.go:276] 2 containers: [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4]
I0307 19:36:10.850923 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.854985 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.858671 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0307 19:36:10.858755 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0307 19:36:10.909124 762831 cri.go:89] found id: "eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:10.909149 762831 cri.go:89] found id: "5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:10.909155 762831 cri.go:89] found id: ""
I0307 19:36:10.909162 762831 logs.go:276] 2 containers: [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907]
I0307 19:36:10.909233 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.913385 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.917239 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0307 19:36:10.917311 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0307 19:36:10.962456 762831 cri.go:89] found id: "2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:10.962475 762831 cri.go:89] found id: "f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:10.962480 762831 cri.go:89] found id: ""
I0307 19:36:10.962487 762831 logs.go:276] 2 containers: [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08]
I0307 19:36:10.962547 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.966970 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:10.970855 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0307 19:36:10.970976 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0307 19:36:11.016102 762831 cri.go:89] found id: "18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:11.016128 762831 cri.go:89] found id: "1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:11.016134 762831 cri.go:89] found id: ""
I0307 19:36:11.016141 762831 logs.go:276] 2 containers: [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29]
I0307 19:36:11.016221 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.019977 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.023680 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0307 19:36:11.023815 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0307 19:36:11.062030 762831 cri.go:89] found id: "ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:11.062103 762831 cri.go:89] found id: "f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:11.062141 762831 cri.go:89] found id: ""
I0307 19:36:11.062151 762831 logs.go:276] 2 containers: [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093]
I0307 19:36:11.062215 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.066359 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.070205 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0307 19:36:11.070279 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0307 19:36:11.109710 762831 cri.go:89] found id: "90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:11.109783 762831 cri.go:89] found id: ""
I0307 19:36:11.109813 762831 logs.go:276] 1 containers: [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a]
I0307 19:36:11.109911 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:11.114323 762831 logs.go:123] Gathering logs for kube-apiserver [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020] ...
I0307 19:36:11.114404 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:11.176440 762831 logs.go:123] Gathering logs for etcd [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d] ...
I0307 19:36:11.176475 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:11.230993 762831 logs.go:123] Gathering logs for kube-scheduler [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f] ...
I0307 19:36:11.231024 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:11.300327 762831 logs.go:123] Gathering logs for kube-proxy [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5] ...
I0307 19:36:11.300359 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:07.375251 771734 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap
I0307 19:36:07.392229 771734 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.28.4/kubectl ...
I0307 19:36:07.392292 771734 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes)
I0307 19:36:07.423731 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0307 19:36:08.486857 771734 ssh_runner.go:235] Completed: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: (1.063085403s)
I0307 19:36:08.486899 771734 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0307 19:36:08.487028 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:08.487123 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes embed-certs-327564 minikube.k8s.io/updated_at=2024_03_07T19_36_08_0700 minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=526fad16cb967ea3a5b243df32efb88cb58b81ec minikube.k8s.io/name=embed-certs-327564 minikube.k8s.io/primary=true
I0307 19:36:08.497308 771734 ops.go:34] apiserver oom_adj: -16
I0307 19:36:08.710034 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:09.210820 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:09.711072 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:10.210737 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:10.710463 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:11.210338 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:11.710103 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:11.354977 762831 logs.go:123] Gathering logs for storage-provisioner [f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093] ...
I0307 19:36:11.355007 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:11.398803 762831 logs.go:123] Gathering logs for containerd ...
I0307 19:36:11.398834 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0307 19:36:11.458507 762831 logs.go:123] Gathering logs for kube-controller-manager [f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08] ...
I0307 19:36:11.458543 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:11.532204 762831 logs.go:123] Gathering logs for kubernetes-dashboard [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a] ...
I0307 19:36:11.535311 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:11.601998 762831 logs.go:123] Gathering logs for describe nodes ...
I0307 19:36:11.602025 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0307 19:36:11.766962 762831 logs.go:123] Gathering logs for kube-apiserver [99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939] ...
I0307 19:36:11.766998 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:11.869356 762831 logs.go:123] Gathering logs for coredns [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b] ...
I0307 19:36:11.869391 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:11.909582 762831 logs.go:123] Gathering logs for coredns [f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384] ...
I0307 19:36:11.909612 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:11.955107 762831 logs.go:123] Gathering logs for kube-proxy [5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907] ...
I0307 19:36:11.955137 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:11.995979 762831 logs.go:123] Gathering logs for kube-controller-manager [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916] ...
I0307 19:36:11.996008 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:12.056706 762831 logs.go:123] Gathering logs for dmesg ...
I0307 19:36:12.056737 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0307 19:36:12.076984 762831 logs.go:123] Gathering logs for kindnet [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57] ...
I0307 19:36:12.077015 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:12.120010 762831 logs.go:123] Gathering logs for container status ...
I0307 19:36:12.120041 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0307 19:36:12.163174 762831 logs.go:123] Gathering logs for kubelet ...
I0307 19:36:12.163205 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0307 19:36:12.218431 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562011 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-vdtz2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vdtz2" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.218682 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562119 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.218920 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562202 662 reflector.go:138] object-"kube-system"/"coredns-token-zckzk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zckzk" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.219145 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562259 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220131 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562331 662 reflector.go:138] object-"kube-system"/"kindnet-token-g7knv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g7knv" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220392 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562675 662 reflector.go:138] object-"default"/"default-token-rgss9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-rgss9" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220637 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562749 662 reflector.go:138] object-"kube-system"/"metrics-server-token-t79rr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-t79rr" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.220886 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.672443 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k4nsq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k4nsq" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:12.233667 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.291729 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.233913 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.749946 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.236687 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:07 old-k8s-version-490121 kubelet[662]: E0307 19:31:07.413698 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.238386 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:20 old-k8s-version-490121 kubelet[662]: E0307 19:31:20.430619 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.239318 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:23 old-k8s-version-490121 kubelet[662]: E0307 19:31:23.898812 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.239668 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:24 old-k8s-version-490121 kubelet[662]: E0307 19:31:24.894861 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.240124 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:25 old-k8s-version-490121 kubelet[662]: E0307 19:31:25.899230 662 pod_workers.go:191] Error syncing pod 57707985-e3ad-463c-a7a9-150bfe271af7 ("storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"
W0307 19:36:12.242571 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:31 old-k8s-version-490121 kubelet[662]: E0307 19:31:31.411639 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.242927 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:32 old-k8s-version-490121 kubelet[662]: E0307 19:31:32.839916 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.243723 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.406752 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.244204 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.954053 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.244574 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:52 old-k8s-version-490121 kubelet[662]: E0307 19:31:52.840518 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.244771 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:57 old-k8s-version-490121 kubelet[662]: E0307 19:31:57.404922 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.245369 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.021627 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.249632 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.404313 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.249991 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:12 old-k8s-version-490121 kubelet[662]: E0307 19:32:12.840494 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.252464 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:19 old-k8s-version-490121 kubelet[662]: E0307 19:32:19.412896 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.252795 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:25 old-k8s-version-490121 kubelet[662]: E0307 19:32:25.404097 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.252980 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:34 old-k8s-version-490121 kubelet[662]: E0307 19:32:34.404539 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.253309 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:38 old-k8s-version-490121 kubelet[662]: E0307 19:32:38.404439 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.253495 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:49 old-k8s-version-490121 kubelet[662]: E0307 19:32:49.404401 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.254222 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:51 old-k8s-version-490121 kubelet[662]: E0307 19:32:51.130308 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.254581 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:52 old-k8s-version-490121 kubelet[662]: E0307 19:32:52.840092 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.254790 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:02 old-k8s-version-490121 kubelet[662]: E0307 19:33:02.404637 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.255142 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:05 old-k8s-version-490121 kubelet[662]: E0307 19:33:05.404490 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.255352 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:14 old-k8s-version-490121 kubelet[662]: E0307 19:33:14.404856 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.255709 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:17 old-k8s-version-490121 kubelet[662]: E0307 19:33:17.403976 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.255916 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:27 old-k8s-version-490121 kubelet[662]: E0307 19:33:27.404480 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.256274 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:30 old-k8s-version-490121 kubelet[662]: E0307 19:33:30.404449 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.256625 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:41 old-k8s-version-490121 kubelet[662]: E0307 19:33:41.404057 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.260957 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:42 old-k8s-version-490121 kubelet[662]: E0307 19:33:42.415836 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:12.261998 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:52 old-k8s-version-490121 kubelet[662]: E0307 19:33:52.404978 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.262221 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:55 old-k8s-version-490121 kubelet[662]: E0307 19:33:55.404850 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.264387 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:07 old-k8s-version-490121 kubelet[662]: E0307 19:34:07.404060 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.264606 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:09 old-k8s-version-490121 kubelet[662]: E0307 19:34:09.404722 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.265123 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:22 old-k8s-version-490121 kubelet[662]: E0307 19:34:22.325297 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.266214 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.328654 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.266457 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.404300 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.266819 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:34 old-k8s-version-490121 kubelet[662]: E0307 19:34:34.405645 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.267028 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:35 old-k8s-version-490121 kubelet[662]: E0307 19:34:35.404699 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.267382 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:47 old-k8s-version-490121 kubelet[662]: E0307 19:34:47.404007 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.267620 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:49 old-k8s-version-490121 kubelet[662]: E0307 19:34:49.404434 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.267992 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:59 old-k8s-version-490121 kubelet[662]: E0307 19:34:59.404258 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.268205 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:02 old-k8s-version-490121 kubelet[662]: E0307 19:35:02.405853 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.268570 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: E0307 19:35:12.405265 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.268783 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:17 old-k8s-version-490121 kubelet[662]: E0307 19:35:17.404385 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.269134 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: E0307 19:35:27.406835 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.269703 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:32 old-k8s-version-490121 kubelet[662]: E0307 19:35:32.407052 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.270066 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: E0307 19:35:40.404052 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.270279 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.270632 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.270841 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.271203 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.271558 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:12.271580 762831 logs.go:123] Gathering logs for etcd [05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190] ...
I0307 19:36:12.271605 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:12.320083 762831 logs.go:123] Gathering logs for kube-scheduler [156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4] ...
I0307 19:36:12.320115 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:12.372179 762831 logs.go:123] Gathering logs for kindnet [1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29] ...
I0307 19:36:12.372386 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:12.424384 762831 logs.go:123] Gathering logs for storage-provisioner [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa] ...
I0307 19:36:12.424413 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:12.473228 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:12.473254 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0307 19:36:12.473310 762831 out.go:239] X Problems detected in kubelet:
W0307 19:36:12.473329 762831 out.go:239] Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.473339 762831 out.go:239] Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.473348 762831 out.go:239] Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:12.473363 762831 out.go:239] Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:12.473369 762831 out.go:239] Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:12.473377 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:12.473382 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:36:12.210091 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:12.710748 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:13.210547 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:13.710781 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:14.210651 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:14.710863 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:15.210137 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:15.710393 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:16.210113 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:16.710723 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:17.210816 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:17.710738 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:18.210369 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:18.710992 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:19.210261 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:19.710972 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:20.210057 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:20.710270 771734 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0307 19:36:20.817387 771734 kubeadm.go:1106] duration metric: took 12.330406209s to wait for elevateKubeSystemPrivileges
W0307 19:36:20.817424 771734 kubeadm.go:286] apiserver tunnel failed: apiserver port not set
I0307 19:36:20.817433 771734 kubeadm.go:393] duration metric: took 32.24545592s to StartCluster
I0307 19:36:20.817450 771734 settings.go:142] acquiring lock: {Name:mkebfa804b6349436c6d99572f0f0da9cb5ad1f6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:36:20.817550 771734 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18239-558171/kubeconfig
I0307 19:36:20.818982 771734 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18239-558171/kubeconfig: {Name:mk6862a934ece36327360ff645a33ee6e04a2f89 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0307 19:36:20.819199 771734 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0307 19:36:20.821512 771734 out.go:177] * Verifying Kubernetes components...
I0307 19:36:20.819321 771734 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0307 19:36:20.819484 771734 config.go:182] Loaded profile config "embed-certs-327564": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.28.4
I0307 19:36:20.819493 771734 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0307 19:36:20.823483 771734 addons.go:69] Setting storage-provisioner=true in profile "embed-certs-327564"
I0307 19:36:20.823509 771734 addons.go:234] Setting addon storage-provisioner=true in "embed-certs-327564"
I0307 19:36:20.823515 771734 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0307 19:36:20.823538 771734 host.go:66] Checking if "embed-certs-327564" exists ...
I0307 19:36:20.823689 771734 addons.go:69] Setting default-storageclass=true in profile "embed-certs-327564"
I0307 19:36:20.823710 771734 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "embed-certs-327564"
I0307 19:36:20.823976 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:36:20.824029 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:36:20.857965 771734 addons.go:234] Setting addon default-storageclass=true in "embed-certs-327564"
I0307 19:36:20.858005 771734 host.go:66] Checking if "embed-certs-327564" exists ...
I0307 19:36:20.858433 771734 cli_runner.go:164] Run: docker container inspect embed-certs-327564 --format={{.State.Status}}
I0307 19:36:20.870983 771734 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0307 19:36:20.873019 771734 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:36:20.873043 771734 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0307 19:36:20.873114 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:36:20.898501 771734 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0307 19:36:20.898522 771734 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0307 19:36:20.898586 771734 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-327564
I0307 19:36:20.913533 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:36:20.932599 771734 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33818 SSHKeyPath:/home/jenkins/minikube-integration/18239-558171/.minikube/machines/embed-certs-327564/id_rsa Username:docker}
I0307 19:36:21.035830 771734 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0307 19:36:21.132514 771734 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.76.1 host.minikube.internal\n fallthrough\n }' -e '/^ errors *$/i \ log' | sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0307 19:36:21.133934 771734 node_ready.go:35] waiting up to 6m0s for node "embed-certs-327564" to be "Ready" ...
I0307 19:36:21.153599 771734 node_ready.go:49] node "embed-certs-327564" has status "Ready":"True"
I0307 19:36:21.153677 771734 node_ready.go:38] duration metric: took 19.657606ms for node "embed-certs-327564" to be "Ready" ...
I0307 19:36:21.153710 771734 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0307 19:36:21.162737 771734 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-7bbqp" in "kube-system" namespace to be "Ready" ...
I0307 19:36:21.195702 771734 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0307 19:36:21.224311 771734 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0307 19:36:21.730101 771734 start.go:948] {"host.minikube.internal": 192.168.76.1} host record injected into CoreDNS's ConfigMap
I0307 19:36:22.120906 771734 out.go:177] * Enabled addons: default-storageclass, storage-provisioner
I0307 19:36:22.474361 762831 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0307 19:36:22.494422 762831 api_server.go:72] duration metric: took 5m48.072661998s to wait for apiserver process to appear ...
I0307 19:36:22.494445 762831 api_server.go:88] waiting for apiserver healthz status ...
I0307 19:36:22.494486 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0307 19:36:22.494542 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0307 19:36:22.559595 762831 cri.go:89] found id: "ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:22.559616 762831 cri.go:89] found id: "99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:22.559624 762831 cri.go:89] found id: ""
I0307 19:36:22.559632 762831 logs.go:276] 2 containers: [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939]
I0307 19:36:22.559686 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.563850 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.568388 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0307 19:36:22.568464 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0307 19:36:22.628461 762831 cri.go:89] found id: "5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:22.628491 762831 cri.go:89] found id: "05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:22.628496 762831 cri.go:89] found id: ""
I0307 19:36:22.628504 762831 logs.go:276] 2 containers: [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190]
I0307 19:36:22.628558 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.632518 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.636566 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0307 19:36:22.636648 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0307 19:36:22.716216 762831 cri.go:89] found id: "9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:22.716236 762831 cri.go:89] found id: "f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:22.716241 762831 cri.go:89] found id: ""
I0307 19:36:22.716248 762831 logs.go:276] 2 containers: [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384]
I0307 19:36:22.716303 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.720114 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.726286 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0307 19:36:22.726389 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0307 19:36:22.775384 762831 cri.go:89] found id: "62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:22.775417 762831 cri.go:89] found id: "156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:22.775423 762831 cri.go:89] found id: ""
I0307 19:36:22.775431 762831 logs.go:276] 2 containers: [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4]
I0307 19:36:22.775535 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.779574 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.783105 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0307 19:36:22.783214 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0307 19:36:22.834745 762831 cri.go:89] found id: "eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:22.834766 762831 cri.go:89] found id: "5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:22.834771 762831 cri.go:89] found id: ""
I0307 19:36:22.834778 762831 logs.go:276] 2 containers: [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907]
I0307 19:36:22.834885 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.838940 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.842643 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0307 19:36:22.842766 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0307 19:36:22.885986 762831 cri.go:89] found id: "2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:22.886051 762831 cri.go:89] found id: "f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:22.886073 762831 cri.go:89] found id: ""
I0307 19:36:22.886087 762831 logs.go:276] 2 containers: [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08]
I0307 19:36:22.886147 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.889768 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.893174 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0307 19:36:22.893250 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0307 19:36:22.932715 762831 cri.go:89] found id: "18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:22.932734 762831 cri.go:89] found id: "1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:22.932739 762831 cri.go:89] found id: ""
I0307 19:36:22.932746 762831 logs.go:276] 2 containers: [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29]
I0307 19:36:22.932801 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.936637 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.940288 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0307 19:36:22.940389 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0307 19:36:22.982459 762831 cri.go:89] found id: "ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:22.982483 762831 cri.go:89] found id: "f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:22.982488 762831 cri.go:89] found id: ""
I0307 19:36:22.982495 762831 logs.go:276] 2 containers: [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093]
I0307 19:36:22.982568 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.986251 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:22.989380 762831 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0307 19:36:22.989460 762831 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0307 19:36:23.030709 762831 cri.go:89] found id: "90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:23.030732 762831 cri.go:89] found id: ""
I0307 19:36:23.030739 762831 logs.go:276] 1 containers: [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a]
I0307 19:36:23.030816 762831 ssh_runner.go:195] Run: which crictl
I0307 19:36:23.034771 762831 logs.go:123] Gathering logs for coredns [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b] ...
I0307 19:36:23.034798 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b"
I0307 19:36:23.078994 762831 logs.go:123] Gathering logs for kube-scheduler [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f] ...
I0307 19:36:23.079022 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f"
I0307 19:36:23.126348 762831 logs.go:123] Gathering logs for kube-proxy [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5] ...
I0307 19:36:23.126376 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5"
I0307 19:36:23.172310 762831 logs.go:123] Gathering logs for storage-provisioner [f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093] ...
I0307 19:36:23.172375 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093"
I0307 19:36:23.212617 762831 logs.go:123] Gathering logs for containerd ...
I0307 19:36:23.212684 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0307 19:36:23.282162 762831 logs.go:123] Gathering logs for kubelet ...
I0307 19:36:23.282199 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0307 19:36:23.342202 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562011 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-vdtz2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vdtz2" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.342735 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562119 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343078 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562202 662 reflector.go:138] object-"kube-system"/"coredns-token-zckzk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zckzk" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343584 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562259 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.343891 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562331 662 reflector.go:138] object-"kube-system"/"kindnet-token-g7knv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g7knv" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.344406 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562675 662 reflector.go:138] object-"default"/"default-token-rgss9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-rgss9" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.344748 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.562749 662 reflector.go:138] object-"kube-system"/"metrics-server-token-t79rr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-t79rr" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.345285 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:51 old-k8s-version-490121 kubelet[662]: E0307 19:30:51.672443 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k4nsq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k4nsq" is forbidden: User "system:node:old-k8s-version-490121" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-490121' and this object
W0307 19:36:23.353834 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.291729 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.354037 762831 logs.go:138] Found kubelet problem: Mar 07 19:30:54 old-k8s-version-490121 kubelet[662]: E0307 19:30:54.749946 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.356811 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:07 old-k8s-version-490121 kubelet[662]: E0307 19:31:07.413698 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.358512 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:20 old-k8s-version-490121 kubelet[662]: E0307 19:31:20.430619 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.359433 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:23 old-k8s-version-490121 kubelet[662]: E0307 19:31:23.898812 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.359765 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:24 old-k8s-version-490121 kubelet[662]: E0307 19:31:24.894861 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.360200 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:25 old-k8s-version-490121 kubelet[662]: E0307 19:31:25.899230 662 pod_workers.go:191] Error syncing pod 57707985-e3ad-463c-a7a9-150bfe271af7 ("storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(57707985-e3ad-463c-a7a9-150bfe271af7)"
W0307 19:36:23.364160 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:31 old-k8s-version-490121 kubelet[662]: E0307 19:31:31.411639 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.364506 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:32 old-k8s-version-490121 kubelet[662]: E0307 19:31:32.839916 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.365283 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.406752 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.365814 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:43 old-k8s-version-490121 kubelet[662]: E0307 19:31:43.954053 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.366145 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:52 old-k8s-version-490121 kubelet[662]: E0307 19:31:52.840518 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.366334 762831 logs.go:138] Found kubelet problem: Mar 07 19:31:57 old-k8s-version-490121 kubelet[662]: E0307 19:31:57.404922 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.366928 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.021627 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.367115 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:08 old-k8s-version-490121 kubelet[662]: E0307 19:32:08.404313 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.367441 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:12 old-k8s-version-490121 kubelet[662]: E0307 19:32:12.840494 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.369872 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:19 old-k8s-version-490121 kubelet[662]: E0307 19:32:19.412896 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.370699 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:25 old-k8s-version-490121 kubelet[662]: E0307 19:32:25.404097 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.370901 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:34 old-k8s-version-490121 kubelet[662]: E0307 19:32:34.404539 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.371238 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:38 old-k8s-version-490121 kubelet[662]: E0307 19:32:38.404439 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.371423 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:49 old-k8s-version-490121 kubelet[662]: E0307 19:32:49.404401 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.372010 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:51 old-k8s-version-490121 kubelet[662]: E0307 19:32:51.130308 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.372334 762831 logs.go:138] Found kubelet problem: Mar 07 19:32:52 old-k8s-version-490121 kubelet[662]: E0307 19:32:52.840092 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.372514 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:02 old-k8s-version-490121 kubelet[662]: E0307 19:33:02.404637 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.372838 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:05 old-k8s-version-490121 kubelet[662]: E0307 19:33:05.404490 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.373020 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:14 old-k8s-version-490121 kubelet[662]: E0307 19:33:14.404856 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.373343 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:17 old-k8s-version-490121 kubelet[662]: E0307 19:33:17.403976 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.373536 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:27 old-k8s-version-490121 kubelet[662]: E0307 19:33:27.404480 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.373887 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:30 old-k8s-version-490121 kubelet[662]: E0307 19:33:30.404449 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.374213 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:41 old-k8s-version-490121 kubelet[662]: E0307 19:33:41.404057 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.376719 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:42 old-k8s-version-490121 kubelet[662]: E0307 19:33:42.415836 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0307 19:36:23.377054 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:52 old-k8s-version-490121 kubelet[662]: E0307 19:33:52.404978 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.377238 762831 logs.go:138] Found kubelet problem: Mar 07 19:33:55 old-k8s-version-490121 kubelet[662]: E0307 19:33:55.404850 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.377572 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:07 old-k8s-version-490121 kubelet[662]: E0307 19:34:07.404060 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.377754 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:09 old-k8s-version-490121 kubelet[662]: E0307 19:34:09.404722 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.378206 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:22 old-k8s-version-490121 kubelet[662]: E0307 19:34:22.325297 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.378658 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.328654 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.378843 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:23 old-k8s-version-490121 kubelet[662]: E0307 19:34:23.404300 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.379171 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:34 old-k8s-version-490121 kubelet[662]: E0307 19:34:34.405645 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.379355 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:35 old-k8s-version-490121 kubelet[662]: E0307 19:34:35.404699 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.379679 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:47 old-k8s-version-490121 kubelet[662]: E0307 19:34:47.404007 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.379860 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:49 old-k8s-version-490121 kubelet[662]: E0307 19:34:49.404434 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.380186 762831 logs.go:138] Found kubelet problem: Mar 07 19:34:59 old-k8s-version-490121 kubelet[662]: E0307 19:34:59.404258 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.380367 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:02 old-k8s-version-490121 kubelet[662]: E0307 19:35:02.405853 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.380716 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: E0307 19:35:12.405265 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.380901 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:17 old-k8s-version-490121 kubelet[662]: E0307 19:35:17.404385 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.381224 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: E0307 19:35:27.406835 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.381405 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:32 old-k8s-version-490121 kubelet[662]: E0307 19:35:32.407052 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.381735 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: E0307 19:35:40.404052 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.382360 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.382702 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.382887 762831 logs.go:138] Found kubelet problem: Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.383217 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.383399 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:23.383723 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:23.383904 762831 logs.go:138] Found kubelet problem: Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:23.383915 762831 logs.go:123] Gathering logs for kube-apiserver [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020] ...
I0307 19:36:23.383933 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020"
I0307 19:36:23.476177 762831 logs.go:123] Gathering logs for kube-controller-manager [f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08] ...
I0307 19:36:23.476227 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08"
I0307 19:36:23.572070 762831 logs.go:123] Gathering logs for storage-provisioner [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa] ...
I0307 19:36:23.572107 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa"
I0307 19:36:23.617823 762831 logs.go:123] Gathering logs for kubernetes-dashboard [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a] ...
I0307 19:36:23.617853 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a"
I0307 19:36:23.696040 762831 logs.go:123] Gathering logs for container status ...
I0307 19:36:23.696108 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0307 19:36:23.762456 762831 logs.go:123] Gathering logs for kube-apiserver [99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939] ...
I0307 19:36:23.762532 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939"
I0307 19:36:23.835674 762831 logs.go:123] Gathering logs for etcd [05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190] ...
I0307 19:36:23.835750 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190"
I0307 19:36:23.903568 762831 logs.go:123] Gathering logs for coredns [f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384] ...
I0307 19:36:23.903699 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384"
I0307 19:36:23.952099 762831 logs.go:123] Gathering logs for kindnet [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57] ...
I0307 19:36:23.952172 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57"
I0307 19:36:24.014266 762831 logs.go:123] Gathering logs for kindnet [1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29] ...
I0307 19:36:24.014344 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29"
I0307 19:36:24.149466 762831 logs.go:123] Gathering logs for describe nodes ...
I0307 19:36:24.149546 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0307 19:36:24.364732 762831 logs.go:123] Gathering logs for etcd [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d] ...
I0307 19:36:24.364896 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d"
I0307 19:36:24.422381 762831 logs.go:123] Gathering logs for kube-proxy [5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907] ...
I0307 19:36:24.422452 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907"
I0307 19:36:24.472397 762831 logs.go:123] Gathering logs for kube-controller-manager [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916] ...
I0307 19:36:24.472461 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916"
I0307 19:36:24.554174 762831 logs.go:123] Gathering logs for dmesg ...
I0307 19:36:24.554248 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0307 19:36:24.577804 762831 logs.go:123] Gathering logs for kube-scheduler [156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4] ...
I0307 19:36:24.577881 762831 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4"
I0307 19:36:24.642561 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:24.642626 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0307 19:36:24.642701 762831 out.go:239] X Problems detected in kubelet:
W0307 19:36:24.642745 762831 out.go:239] Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:24.642899 762831 out.go:239] Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:24.642937 762831 out.go:239] Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0307 19:36:24.643025 762831 out.go:239] Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
W0307 19:36:24.643060 762831 out.go:239] Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0307 19:36:24.643094 762831 out.go:304] Setting ErrFile to fd 2...
I0307 19:36:24.643117 762831 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0307 19:36:22.123372 771734 addons.go:505] duration metric: took 1.303869004s for enable addons: enabled=[default-storageclass storage-provisioner]
I0307 19:36:22.234856 771734 kapi.go:248] "coredns" deployment in "kube-system" namespace and "embed-certs-327564" context rescaled to 1 replicas
I0307 19:36:23.170538 771734 pod_ready.go:102] pod "coredns-5dd5756b68-7bbqp" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:25.669208 771734 pod_ready.go:102] pod "coredns-5dd5756b68-7bbqp" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:27.669815 771734 pod_ready.go:102] pod "coredns-5dd5756b68-7bbqp" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:30.170553 771734 pod_ready.go:102] pod "coredns-5dd5756b68-7bbqp" in "kube-system" namespace has status "Ready":"False"
I0307 19:36:34.644283 762831 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0307 19:36:34.654169 762831 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0307 19:36:34.656597 762831 out.go:177]
W0307 19:36:34.658535 762831 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0307 19:36:34.658604 762831 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0307 19:36:34.658626 762831 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0307 19:36:34.658632 762831 out.go:239] *
W0307 19:36:34.659694 762831 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0307 19:36:34.662993 762831 out.go:177]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
a6084cdde944f 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 5dc0d6ef12e44 dashboard-metrics-scraper-8d5bb5db8-w2dn4
ef7f1755441a0 ba04bb24b9575 4 minutes ago Running storage-provisioner 2 63cf698ab6bf4 storage-provisioner
90801d79bb21b 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 a888f023f13f0 kubernetes-dashboard-cd95d586-pqw67
d0441d0ae48e3 1611cd07b61d5 5 minutes ago Running busybox 1 84448e0067aab busybox
f1273e5fc5a03 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 63cf698ab6bf4 storage-provisioner
eef127c3b8686 25a5233254979 5 minutes ago Running kube-proxy 1 9e15124462ded kube-proxy-5rbpn
18ade5d6b5c4c 4740c1948d3fc 5 minutes ago Running kindnet-cni 1 e1c85816835c9 kindnet-mgtjt
9a2f527ecba34 db91994f4ee8f 5 minutes ago Running coredns 1 797b3f8a9f4e8 coredns-74ff55c5b-mskrg
ca7ce18516e00 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 28ef2f2ba5200 kube-apiserver-old-k8s-version-490121
62592ab3b7ee4 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 b06031c31470a kube-scheduler-old-k8s-version-490121
2ab85d0a64427 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 2061ba7118f7b kube-controller-manager-old-k8s-version-490121
5d6a1009f3a73 05b738aa1bc63 5 minutes ago Running etcd 1 5694f449f4130 etcd-old-k8s-version-490121
378a2b8295ca6 1611cd07b61d5 6 minutes ago Exited busybox 0 dc5ecff6c4b62 busybox
f4e00b3f8611c db91994f4ee8f 7 minutes ago Exited coredns 0 dcb87b2351872 coredns-74ff55c5b-mskrg
5c8bdc8e9ddfa 25a5233254979 8 minutes ago Exited kube-proxy 0 18120dda8fd8d kube-proxy-5rbpn
1a64bbfdc2b02 4740c1948d3fc 8 minutes ago Exited kindnet-cni 0 097e98a4a65f4 kindnet-mgtjt
99dfdaab36b9d 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 40d81581b981a kube-apiserver-old-k8s-version-490121
156315217b9e0 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 73d02cf9f767b kube-scheduler-old-k8s-version-490121
f425b02ac359c 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 96d681c6ef23e kube-controller-manager-old-k8s-version-490121
05b5946f07423 05b738aa1bc63 8 minutes ago Exited etcd 0 10accc58b0d6b etcd-old-k8s-version-490121
==> containerd <==
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.440571894Z" level=info msg="CreateContainer within sandbox \"5dc0d6ef12e443a853686be440a14e209f52876f39b35e7d1015d003238195a5\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:4,} returns container id \"0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64\""
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.442533508Z" level=info msg="StartContainer for \"0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64\""
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.506671313Z" level=info msg="StartContainer for \"0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64\" returns successfully"
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.536232803Z" level=info msg="shim disconnected" id=0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.536455112Z" level=warning msg="cleaning up after shim disconnected" id=0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64 namespace=k8s.io
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.536483953Z" level=info msg="cleaning up dead shim"
Mar 07 19:32:50 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:50.544851062Z" level=warning msg="cleanup warnings time=\"2024-03-07T19:32:50Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=2936 runtime=io.containerd.runc.v2\n"
Mar 07 19:32:51 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:51.132138052Z" level=info msg="RemoveContainer for \"9124327f2d97792e01774b6641723105520a1b2505cae69e68b96ed4b201e36c\""
Mar 07 19:32:51 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:32:51.147697074Z" level=info msg="RemoveContainer for \"9124327f2d97792e01774b6641723105520a1b2505cae69e68b96ed4b201e36c\" returns successfully"
Mar 07 19:33:42 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:33:42.405636840Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:33:42 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:33:42.412435129Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Mar 07 19:33:42 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:33:42.414277589Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.406926154Z" level=info msg="CreateContainer within sandbox \"5dc0d6ef12e443a853686be440a14e209f52876f39b35e7d1015d003238195a5\" for container &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,}"
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.423134820Z" level=info msg="CreateContainer within sandbox \"5dc0d6ef12e443a853686be440a14e209f52876f39b35e7d1015d003238195a5\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,} returns container id \"a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631\""
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.424031275Z" level=info msg="StartContainer for \"a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631\""
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.486547955Z" level=info msg="StartContainer for \"a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631\" returns successfully"
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.512644362Z" level=info msg="shim disconnected" id=a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.512720440Z" level=warning msg="cleaning up after shim disconnected" id=a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631 namespace=k8s.io
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.512734093Z" level=info msg="cleaning up dead shim"
Mar 07 19:34:21 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:21.521908041Z" level=warning msg="cleanup warnings time=\"2024-03-07T19:34:21Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=3195 runtime=io.containerd.runc.v2\n"
Mar 07 19:34:22 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:22.329707522Z" level=info msg="RemoveContainer for \"0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64\""
Mar 07 19:34:22 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:34:22.335280776Z" level=info msg="RemoveContainer for \"0582a2756b43a22a7724205dce09f283b2c6e2e195a3891d5f3b3bbdd61ced64\" returns successfully"
Mar 07 19:36:35 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:36:35.405322296Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:35 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:36:35.421868846Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Mar 07 19:36:35 old-k8s-version-490121 containerd[568]: time="2024-03-07T19:36:35.423941959Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
==> coredns [9a2f527ecba342bae8ed8f07e3dacc6fe618fe87d31fa4c8f552ebffaa55599b] <==
I0307 19:31:24.683345 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-07 19:30:54.682736585 +0000 UTC m=+0.084033635) (total time: 30.000490878s):
Trace[2019727887]: [30.000490878s] [30.000490878s] END
E0307 19:31:24.683376 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0307 19:31:24.683474 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-07 19:30:54.683241321 +0000 UTC m=+0.084538371) (total time: 30.000221849s):
Trace[939984059]: [30.000221849s] [30.000221849s] END
E0307 19:31:24.683478 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0307 19:31:24.694389 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-07 19:30:54.693912173 +0000 UTC m=+0.095209223) (total time: 30.000448203s):
Trace[1474941318]: [30.000448203s] [30.000448203s] END
E0307 19:31:24.694409 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:34036 - 17 "HINFO IN 3603254449104899921.6502033093172393733. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.022910074s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> coredns [f4e00b3f8611c41774e97ced61d813c222e4c477311f777ff32be35b75f70384] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:45727 - 6069 "HINFO IN 3740224681191150328.849708405113895451. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.02425604s
==> describe nodes <==
Name: old-k8s-version-490121
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-490121
kubernetes.io/os=linux
minikube.k8s.io/commit=526fad16cb967ea3a5b243df32efb88cb58b81ec
minikube.k8s.io/name=old-k8s-version-490121
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_03_07T19_27_53_0700
minikube.k8s.io/version=v1.32.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 07 Mar 2024 19:27:50 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-490121
AcquireTime: <unset>
RenewTime: Thu, 07 Mar 2024 19:36:34 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 07 Mar 2024 19:31:52 +0000 Thu, 07 Mar 2024 19:27:43 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 07 Mar 2024 19:31:52 +0000 Thu, 07 Mar 2024 19:27:43 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 07 Mar 2024 19:31:52 +0000 Thu, 07 Mar 2024 19:27:43 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 07 Mar 2024 19:31:52 +0000 Thu, 07 Mar 2024 19:28:08 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-490121
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022496Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022496Ki
pods: 110
System Info:
Machine ID: facb49c7e4f543eb91788bc786542810
System UUID: 580b3f60-f802-4435-bab4-1dc7cfcb4a04
Boot ID: a949ea88-4a69-4ab0-89c5-986450203265
Kernel Version: 5.15.0-1055-aws
OS Image: Ubuntu 22.04.3 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.6.28
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m33s
kube-system coredns-74ff55c5b-mskrg 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m28s
kube-system etcd-old-k8s-version-490121 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m34s
kube-system kindnet-mgtjt 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m28s
kube-system kube-apiserver-old-k8s-version-490121 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m34s
kube-system kube-controller-manager-old-k8s-version-490121 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m34s
kube-system kube-proxy-5rbpn 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m28s
kube-system kube-scheduler-old-k8s-version-490121 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m34s
kube-system metrics-server-9975d5f86-w9hn6 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m22s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m26s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-w2dn4 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m26s
kubernetes-dashboard kubernetes-dashboard-cd95d586-pqw67 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m26s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m54s (x5 over 8m55s) kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m54s (x4 over 8m55s) kubelet Node old-k8s-version-490121 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m54s (x4 over 8m55s) kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientPID
Normal Starting 8m35s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m35s kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m35s kubelet Node old-k8s-version-490121 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m35s kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m35s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m28s kubelet Node old-k8s-version-490121 status is now: NodeReady
Normal Starting 8m25s kube-proxy Starting kube-proxy.
Normal Starting 5m54s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m54s (x8 over 5m54s) kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m54s (x7 over 5m54s) kubelet Node old-k8s-version-490121 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m54s (x8 over 5m54s) kubelet Node old-k8s-version-490121 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m54s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m41s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.001107] FS-Cache: O-key=[8] 'd83c5c0100000000'
[ +0.000774] FS-Cache: N-cookie c=00000030 [p=00000027 fl=2 nc=0 na=1]
[ +0.001034] FS-Cache: N-cookie d=0000000083f1d117{9p.inode} n=00000000bb5cdba7
[ +0.001104] FS-Cache: N-key=[8] 'd83c5c0100000000'
[ +0.003880] FS-Cache: Duplicate cookie detected
[ +0.000717] FS-Cache: O-cookie c=0000002a [p=00000027 fl=226 nc=0 na=1]
[ +0.000977] FS-Cache: O-cookie d=0000000083f1d117{9p.inode} n=00000000860da42d
[ +0.001206] FS-Cache: O-key=[8] 'd83c5c0100000000'
[ +0.000720] FS-Cache: N-cookie c=00000031 [p=00000027 fl=2 nc=0 na=1]
[ +0.000969] FS-Cache: N-cookie d=0000000083f1d117{9p.inode} n=000000002d1b8011
[ +0.001027] FS-Cache: N-key=[8] 'd83c5c0100000000'
[ +2.780951] FS-Cache: Duplicate cookie detected
[ +0.000732] FS-Cache: O-cookie c=00000028 [p=00000027 fl=226 nc=0 na=1]
[ +0.001109] FS-Cache: O-cookie d=0000000083f1d117{9p.inode} n=000000003a76bb30
[ +0.001108] FS-Cache: O-key=[8] 'd73c5c0100000000'
[ +0.000779] FS-Cache: N-cookie c=00000033 [p=00000027 fl=2 nc=0 na=1]
[ +0.000945] FS-Cache: N-cookie d=0000000083f1d117{9p.inode} n=00000000929d8fe2
[ +0.001069] FS-Cache: N-key=[8] 'd73c5c0100000000'
[ +0.306498] FS-Cache: Duplicate cookie detected
[ +0.000774] FS-Cache: O-cookie c=0000002d [p=00000027 fl=226 nc=0 na=1]
[ +0.001003] FS-Cache: O-cookie d=0000000083f1d117{9p.inode} n=0000000010ad09c7
[ +0.001118] FS-Cache: O-key=[8] 'dd3c5c0100000000'
[ +0.000756] FS-Cache: N-cookie c=00000034 [p=00000027 fl=2 nc=0 na=1]
[ +0.000928] FS-Cache: N-cookie d=0000000083f1d117{9p.inode} n=000000002bfb4aa6
[ +0.001033] FS-Cache: N-key=[8] 'dd3c5c0100000000'
==> etcd [05b5946f074233e2a2ab87112eb492b37e819fe68a625e504d3d1683dafbb190] <==
raft2024/03/07 19:27:42 INFO: 9f0758e1c58a86ed received MsgVoteResp from 9f0758e1c58a86ed at term 2
raft2024/03/07 19:27:42 INFO: 9f0758e1c58a86ed became leader at term 2
raft2024/03/07 19:27:42 INFO: raft.node: 9f0758e1c58a86ed elected leader 9f0758e1c58a86ed at term 2
2024-03-07 19:27:42.837755 I | etcdserver: setting up the initial cluster version to 3.4
2024-03-07 19:27:42.838564 I | etcdserver: published {Name:old-k8s-version-490121 ClientURLs:[https://192.168.85.2:2379]} to cluster 68eaea490fab4e05
2024-03-07 19:27:42.838709 I | embed: ready to serve client requests
2024-03-07 19:27:42.842489 I | embed: serving client requests on 192.168.85.2:2379
2024-03-07 19:27:42.842799 I | embed: ready to serve client requests
2024-03-07 19:27:42.846604 I | embed: serving client requests on 127.0.0.1:2379
2024-03-07 19:27:42.852854 N | etcdserver/membership: set the initial cluster version to 3.4
2024-03-07 19:27:42.866567 I | etcdserver/api: enabled capabilities for version 3.4
2024-03-07 19:28:05.809677 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:28:12.388816 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:28:22.388086 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:28:32.388212 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:28:42.391309 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:28:52.388009 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:02.388062 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:12.388220 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:22.388223 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:32.388059 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:42.388082 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:29:52.388181 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:30:02.388192 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:30:12.387944 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [5d6a1009f3a738e6134be28d53df0c689221a6f5c60ec491f9c60aba463a073d] <==
2024-03-07 19:32:29.366319 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:32:39.366346 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:32:49.366477 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:32:59.366387 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:09.366329 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:19.366347 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:29.366392 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:39.366546 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:49.366346 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:33:59.366357 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:09.366370 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:19.366481 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:29.366378 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:39.366843 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:49.366437 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:34:59.366301 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:09.366393 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:19.366703 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:29.366991 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:39.366268 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:49.366458 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:35:59.366382 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:36:09.366492 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:36:19.366462 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-07 19:36:29.366343 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
19:36:36 up 3:19, 0 users, load average: 1.65, 2.00, 2.52
Linux old-k8s-version-490121 5.15.0-1055-aws #60~20.04.1-Ubuntu SMP Thu Feb 22 15:54:21 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.3 LTS"
==> kindnet [18ade5d6b5c4c10eef9993a7987d4939f332c8517182d15c717de0c94fa32e57] <==
I0307 19:34:35.484586 1 main.go:227] handling current node
I0307 19:34:45.500131 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:34:45.500155 1 main.go:227] handling current node
I0307 19:34:55.511003 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:34:55.511030 1 main.go:227] handling current node
I0307 19:35:05.530729 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:05.530772 1 main.go:227] handling current node
I0307 19:35:15.542990 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:15.543022 1 main.go:227] handling current node
I0307 19:35:25.553743 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:25.553837 1 main.go:227] handling current node
I0307 19:35:35.568826 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:35.568859 1 main.go:227] handling current node
I0307 19:35:45.583843 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:45.584269 1 main.go:227] handling current node
I0307 19:35:55.594463 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:35:55.594494 1 main.go:227] handling current node
I0307 19:36:05.616186 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:36:05.616213 1 main.go:227] handling current node
I0307 19:36:15.627573 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:36:15.627601 1 main.go:227] handling current node
I0307 19:36:25.644676 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:36:25.644710 1 main.go:227] handling current node
I0307 19:36:35.662358 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:36:35.662385 1 main.go:227] handling current node
==> kindnet [1a64bbfdc2b02903957dd7d0b87bd907abe69379f055630a01c14b476ce40b29] <==
podIP = 192.168.85.2
I0307 19:28:09.918950 1 main.go:116] setting mtu 1500 for CNI
I0307 19:28:09.918972 1 main.go:146] kindnetd IP family: "ipv4"
I0307 19:28:09.918986 1 main.go:150] noMask IPv4 subnets: [10.244.0.0/16]
I0307 19:28:40.153742 1 main.go:191] Failed to get nodes, retrying after error: Get "https://10.96.0.1:443/api/v1/nodes": dial tcp 10.96.0.1:443: i/o timeout
I0307 19:28:40.167015 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:28:40.167049 1 main.go:227] handling current node
I0307 19:28:50.183769 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:28:50.184937 1 main.go:227] handling current node
I0307 19:29:00.220768 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:00.220868 1 main.go:227] handling current node
I0307 19:29:10.233202 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:10.233783 1 main.go:227] handling current node
I0307 19:29:20.259957 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:20.259990 1 main.go:227] handling current node
I0307 19:29:30.290317 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:30.290427 1 main.go:227] handling current node
I0307 19:29:40.295095 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:40.295431 1 main.go:227] handling current node
I0307 19:29:50.333811 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:29:50.333838 1 main.go:227] handling current node
I0307 19:30:00.415622 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:30:00.415889 1 main.go:227] handling current node
I0307 19:30:10.420304 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0307 19:30:10.420338 1 main.go:227] handling current node
==> kube-apiserver [99dfdaab36b9d1de349d9550c59c8c3f81df15ddb7e0c5b623692865d7a65939] <==
I0307 19:27:50.991336 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0307 19:27:50.991359 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0307 19:27:51.482669 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0307 19:27:51.531207 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0307 19:27:51.673965 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.85.2]
I0307 19:27:51.675039 1 controller.go:606] quota admission added evaluator for: endpoints
I0307 19:27:51.681379 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0307 19:27:52.688168 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0307 19:27:53.226933 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0307 19:27:53.321460 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0307 19:28:01.688944 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0307 19:28:08.916747 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0307 19:28:08.922266 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0307 19:28:25.742778 1 client.go:360] parsed scheme: "passthrough"
I0307 19:28:25.742819 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:28:25.742961 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:28:57.766070 1 client.go:360] parsed scheme: "passthrough"
I0307 19:28:57.766129 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:28:57.766138 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:29:27.818863 1 client.go:360] parsed scheme: "passthrough"
I0307 19:29:27.818911 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:29:27.819064 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:29:59.033907 1 client.go:360] parsed scheme: "passthrough"
I0307 19:29:59.034108 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:29:59.034231 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [ca7ce18516e00288ccaa256e5bd851dd1cd369b890c3a9a183165bdc13389020] <==
I0307 19:33:14.826605 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:33:14.826614 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:33:46.089094 1 client.go:360] parsed scheme: "passthrough"
I0307 19:33:46.089145 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:33:46.089154 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0307 19:33:55.153102 1 handler_proxy.go:102] no RequestInfo found in the context
E0307 19:33:55.153183 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0307 19:33:55.153194 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0307 19:34:25.204004 1 client.go:360] parsed scheme: "passthrough"
I0307 19:34:25.204049 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:34:25.204086 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:35:03.886558 1 client.go:360] parsed scheme: "passthrough"
I0307 19:35:03.886604 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:35:03.886646 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0307 19:35:41.868433 1 client.go:360] parsed scheme: "passthrough"
I0307 19:35:41.868478 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:35:41.868631 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0307 19:35:52.283032 1 handler_proxy.go:102] no RequestInfo found in the context
E0307 19:35:52.283300 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0307 19:35:52.283318 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0307 19:36:11.916104 1 client.go:360] parsed scheme: "passthrough"
I0307 19:36:11.916167 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0307 19:36:11.916176 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [2ab85d0a644279199f2ca1291b3a03fe47b2943a8b80b2d9fc03bc1d42b67916] <==
W0307 19:32:15.836002 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:32:41.889909 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:32:47.486198 1 request.go:655] Throttling request took 1.04741415s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0307 19:32:48.337895 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:33:12.391531 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:33:19.988483 1 request.go:655] Throttling request took 1.048530027s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0307 19:33:20.840002 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:33:42.893416 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:33:52.490428 1 request.go:655] Throttling request took 1.048430389s, request: GET:https://192.168.85.2:8443/apis/autoscaling/v2beta1?timeout=32s
W0307 19:33:53.341832 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:34:13.395391 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:34:24.992405 1 request.go:655] Throttling request took 1.0480377s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0307 19:34:25.843746 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:34:43.897276 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:34:57.494257 1 request.go:655] Throttling request took 1.048473644s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0307 19:34:58.345736 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:35:14.399076 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:35:29.996269 1 request.go:655] Throttling request took 1.039270972s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0307 19:35:30.847885 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:35:44.901273 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:36:02.498538 1 request.go:655] Throttling request took 1.048429751s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0307 19:36:03.350051 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0307 19:36:15.403061 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0307 19:36:35.000600 1 request.go:655] Throttling request took 1.048471659s, request: GET:https://192.168.85.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0307 19:36:35.852904 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [f425b02ac359c5680ff726a835fea08c414b40556ac7ed51243910078d583d08] <==
I0307 19:28:08.918697 1 shared_informer.go:247] Caches are synced for stateful set
I0307 19:28:08.918748 1 shared_informer.go:247] Caches are synced for GC
I0307 19:28:08.918788 1 shared_informer.go:247] Caches are synced for job
I0307 19:28:08.918863 1 shared_informer.go:247] Caches are synced for resource quota
I0307 19:28:08.918930 1 shared_informer.go:247] Caches are synced for taint
I0307 19:28:08.919001 1 node_lifecycle_controller.go:1429] Initializing eviction metric for zone:
W0307 19:28:08.919072 1 node_lifecycle_controller.go:1044] Missing timestamp for Node old-k8s-version-490121. Assuming now as a timestamp.
I0307 19:28:08.919132 1 node_lifecycle_controller.go:1195] Controller detected that all Nodes are not-Ready. Entering master disruption mode.
I0307 19:28:08.919924 1 taint_manager.go:187] Starting NoExecuteTaintManager
I0307 19:28:08.920253 1 event.go:291] "Event occurred" object="old-k8s-version-490121" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-490121 event: Registered Node old-k8s-version-490121 in Controller"
I0307 19:28:08.950365 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0307 19:28:08.975523 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-5rbpn"
I0307 19:28:08.975846 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-mgtjt"
I0307 19:28:08.976846 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-b724r"
I0307 19:28:09.049135 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-mskrg"
I0307 19:28:09.092816 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
E0307 19:28:09.164232 1 daemon_controller.go:320] kube-system/kindnet failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kindnet", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"4d13e347-f351-431f-bb26-082933f88d43", ResourceVersion:"267", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63845436473, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"},\"name\":\"kindnet\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"k
indnet\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"HOST_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.hostIP\"}}},{\"name\":\"POD_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"POD_SUBNET\",\"value\":\"10.244.0.0/16\"}],\"image\":\"docker.io/kindest/kindnetd:v20240202-8f1494ea\",\"name\":\"kindnet-cni\",\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"}},\"securityContext\":{\"capabilities\":{\"add\":[\"NET_RAW\",\"NET_ADMIN\"]},\"privileged\":false},\"volumeMounts\":[{\"mountPath\":\"/etc/cni/net.d\",\"name\":\"cni-cfg\"},{\"mountPath\":\"/run/xtables.lock\",\"name\":\"xtables-lock\",\"readOnly\":false},{\"mountPath\":\"/lib/modules\",\"name\":\"lib-modules\",\"readOnly\":true}]}],\"hostNetwork\":true,\"serviceAccountName\":\"kindnet\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"operator\
":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/etc/cni/net.d\",\"type\":\"DirectoryOrCreate\"},\"name\":\"cni-cfg\"},{\"hostPath\":{\"path\":\"/run/xtables.lock\",\"type\":\"FileOrCreate\"},\"name\":\"xtables-lock\"},{\"hostPath\":{\"path\":\"/lib/modules\"},\"name\":\"lib-modules\"}]}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001bc4940), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001bc4960)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0x4001bc4980), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string
{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"cni-cfg", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001bc49a0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil),
FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001bc49c0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Glust
erfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001bc49e0), EmptyDi
r:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil),
PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kindnet-cni", Image:"docker.io/kindest/kindnetd:v20240202-8f1494ea", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"HOST_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001bc4a00)}, v1.EnvVar{Name:"POD_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001bc4a40)}, v1.EnvVar{Name:"POD_SUBNET", Value:"10.244.0.0/16", ValueFrom:(*v1.EnvVarSource)(nil)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:
0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"cni-cfg", ReadOnly:false, MountPath:"/etc/cni/net.d", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:
(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x4001b09800), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0x4001bd60f8), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"kindnet", DeprecatedServiceAccount:"kindnet", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x400060bb90), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}},
HostAliases:[]v1.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x400070f930)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x4001bd6140)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kindnet": the object has been modified; please apply your changes to the latest version and try again
I0307 19:28:09.360071 1 shared_informer.go:247] Caches are synced for garbage collector
I0307 19:28:09.360109 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0307 19:28:09.393248 1 shared_informer.go:247] Caches are synced for garbage collector
I0307 19:28:10.224939 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0307 19:28:10.237136 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-b724r"
I0307 19:28:13.919400 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0307 19:30:13.621877 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0307 19:30:13.719539 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [5c8bdc8e9ddfad2452883ea6d34d0adcf50e2a71b16cecbe3f4af6d16f9bc907] <==
I0307 19:28:11.425045 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0307 19:28:11.425137 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0307 19:28:11.442547 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0307 19:28:11.442684 1 server_others.go:185] Using iptables Proxier.
I0307 19:28:11.443053 1 server.go:650] Version: v1.20.0
I0307 19:28:11.443866 1 config.go:315] Starting service config controller
I0307 19:28:11.443886 1 shared_informer.go:240] Waiting for caches to sync for service config
I0307 19:28:11.443906 1 config.go:224] Starting endpoint slice config controller
I0307 19:28:11.443966 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0307 19:28:11.544040 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0307 19:28:11.544090 1 shared_informer.go:247] Caches are synced for service config
==> kube-proxy [eef127c3b86862c2bec02399b5a5c139fc6fe1d7d6515b98cdd84fef01157dc5] <==
I0307 19:30:55.335417 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0307 19:30:55.335483 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0307 19:30:55.374020 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0307 19:30:55.374120 1 server_others.go:185] Using iptables Proxier.
I0307 19:30:55.374337 1 server.go:650] Version: v1.20.0
I0307 19:30:55.382515 1 config.go:315] Starting service config controller
I0307 19:30:55.382528 1 shared_informer.go:240] Waiting for caches to sync for service config
I0307 19:30:55.382546 1 config.go:224] Starting endpoint slice config controller
I0307 19:30:55.382549 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0307 19:30:55.482812 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0307 19:30:55.482905 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [156315217b9e0bb9b9b06ef2d546b3e5963161197e29029999b4ae9aac09b2d4] <==
I0307 19:27:45.284494 1 serving.go:331] Generated self-signed cert in-memory
W0307 19:27:50.168590 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0307 19:27:50.168892 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0307 19:27:50.170366 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0307 19:27:50.170673 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0307 19:27:50.252904 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0307 19:27:50.254724 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0307 19:27:50.254957 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0307 19:27:50.257947 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0307 19:27:50.259897 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0307 19:27:50.259998 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0307 19:27:50.260073 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0307 19:27:50.270077 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0307 19:27:50.270426 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0307 19:27:50.270672 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0307 19:27:50.270904 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0307 19:27:50.271187 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0307 19:27:50.271522 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0307 19:27:50.271806 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0307 19:27:50.272101 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0307 19:27:50.272540 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0307 19:27:51.186316 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0307 19:27:51.187595 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0307 19:27:51.224643 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
I0307 19:27:51.758060 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [62592ab3b7ee4a34a9b86dd8976c4e652b42ee4fda964df6ee208432a8e5da1f] <==
I0307 19:30:45.561060 1 serving.go:331] Generated self-signed cert in-memory
W0307 19:30:51.674714 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0307 19:30:51.674945 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0307 19:30:51.675031 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0307 19:30:51.675123 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0307 19:30:51.837457 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0307 19:30:51.837568 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0307 19:30:51.837577 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0307 19:30:51.837589 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0307 19:30:52.046861 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Mar 07 19:35:02 old-k8s-version-490121 kubelet[662]: E0307 19:35:02.405853 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: I0307 19:35:12.403977 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:35:12 old-k8s-version-490121 kubelet[662]: E0307 19:35:12.405265 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:35:17 old-k8s-version-490121 kubelet[662]: E0307 19:35:17.404385 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: I0307 19:35:27.406058 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:35:27 old-k8s-version-490121 kubelet[662]: E0307 19:35:27.406835 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:35:32 old-k8s-version-490121 kubelet[662]: E0307 19:35:32.407052 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: I0307 19:35:40.403721 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:35:40 old-k8s-version-490121 kubelet[662]: E0307 19:35:40.404052 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:35:45 old-k8s-version-490121 kubelet[662]: E0307 19:35:45.404603 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: I0307 19:35:52.404374 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:35:52 old-k8s-version-490121 kubelet[662]: E0307 19:35:52.404806 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:35:56 old-k8s-version-490121 kubelet[662]: E0307 19:35:56.408243 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: I0307 19:36:05.403813 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:36:05 old-k8s-version-490121 kubelet[662]: E0307 19:36:05.404245 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:09 old-k8s-version-490121 kubelet[662]: E0307 19:36:09.404371 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: I0307 19:36:19.403785 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:36:19 old-k8s-version-490121 kubelet[662]: E0307 19:36:19.404863 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:20 old-k8s-version-490121 kubelet[662]: E0307 19:36:20.420688 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 07 19:36:33 old-k8s-version-490121 kubelet[662]: I0307 19:36:33.403706 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a6084cdde944fbe505bcb05e3e41efece4c60e62141bca51f992767826e4f631
Mar 07 19:36:33 old-k8s-version-490121 kubelet[662]: E0307 19:36:33.404056 662 pod_workers.go:191] Error syncing pod 56c3d27a-28f2-422f-83ab-3e31ea3b9a1a ("dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-w2dn4_kubernetes-dashboard(56c3d27a-28f2-422f-83ab-3e31ea3b9a1a)"
Mar 07 19:36:35 old-k8s-version-490121 kubelet[662]: E0307 19:36:35.424322 662 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 07 19:36:35 old-k8s-version-490121 kubelet[662]: E0307 19:36:35.424816 662 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 07 19:36:35 old-k8s-version-490121 kubelet[662]: E0307 19:36:35.425021 662 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-t79rr,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-w9hn6_kube-system(d5b67e4
8-8737-4f11-b097-27b0b334760a): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 07 19:36:35 old-k8s-version-490121 kubelet[662]: E0307 19:36:35.425200 662 pod_workers.go:191] Error syncing pod d5b67e48-8737-4f11-b097-27b0b334760a ("metrics-server-9975d5f86-w9hn6_kube-system(d5b67e48-8737-4f11-b097-27b0b334760a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
==> kubernetes-dashboard [90801d79bb21b5205d18bb8243bac76efccf3c63be39fd6bac993c3caa03646a] <==
2024/03/07 19:31:15 Starting overwatch
2024/03/07 19:31:15 Using namespace: kubernetes-dashboard
2024/03/07 19:31:15 Using in-cluster config to connect to apiserver
2024/03/07 19:31:15 Using secret token for csrf signing
2024/03/07 19:31:15 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/03/07 19:31:15 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/03/07 19:31:16 Successful initial request to the apiserver, version: v1.20.0
2024/03/07 19:31:16 Generating JWE encryption key
2024/03/07 19:31:16 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/03/07 19:31:16 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/03/07 19:31:16 Initializing JWE encryption key from synchronized object
2024/03/07 19:31:16 Creating in-cluster Sidecar client
2024/03/07 19:31:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:31:16 Serving insecurely on HTTP port: 9090
2024/03/07 19:31:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:32:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:32:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:33:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:33:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:34:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:34:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:35:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:35:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/07 19:36:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
==> storage-provisioner [ef7f1755441a0413b4ca24ee824d60b422df9895f6bc51f9b8a913469a14b6aa] <==
I0307 19:31:41.512894 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0307 19:31:41.525281 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0307 19:31:41.525338 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0307 19:31:59.017940 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0307 19:31:59.018186 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"c16ee077-99f9-4aff-9360-fb9b72970221", APIVersion:"v1", ResourceVersion:"855", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-490121_715f41a3-332d-4ebc-bf5f-87f98011376c became leader
I0307 19:31:59.018425 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-490121_715f41a3-332d-4ebc-bf5f-87f98011376c!
I0307 19:31:59.119347 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-490121_715f41a3-332d-4ebc-bf5f-87f98011376c!
==> storage-provisioner [f1273e5fc5a033931d8ba135a0b1fed3cafedc1de1aef7baaffa2a616215e093] <==
I0307 19:30:55.567087 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0307 19:31:25.591278 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-490121 -n old-k8s-version-490121
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-490121 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-w9hn6
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-490121 describe pod metrics-server-9975d5f86-w9hn6
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-490121 describe pod metrics-server-9975d5f86-w9hn6: exit status 1 (104.722947ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-w9hn6" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-490121 describe pod metrics-server-9975d5f86-w9hn6: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (371.82s)