=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-152774 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-152774 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m11.875795896s)
-- stdout --
* [old-k8s-version-152774] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18517
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18517-439728/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18517-439728/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
* Using the docker driver based on existing profile
* Starting "old-k8s-version-152774" primary control-plane node in "old-k8s-version-152774" cluster
* Pulling base image v0.0.43-beta.0 ...
* Restarting existing docker container for "old-k8s-version-152774" ...
* Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-152774 addons enable metrics-server
* Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
-- /stdout --
** stderr **
I0327 19:14:25.883536 642122 out.go:291] Setting OutFile to fd 1 ...
I0327 19:14:25.883718 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:14:25.883730 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:14:25.883743 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:14:25.884038 642122 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18517-439728/.minikube/bin
I0327 19:14:25.884504 642122 out.go:298] Setting JSON to false
I0327 19:14:25.885597 642122 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":10604,"bootTime":1711556262,"procs":203,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1056-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0327 19:14:25.885679 642122 start.go:139] virtualization:
I0327 19:14:25.888177 642122 out.go:177] * [old-k8s-version-152774] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0327 19:14:25.890339 642122 out.go:177] - MINIKUBE_LOCATION=18517
I0327 19:14:25.892112 642122 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0327 19:14:25.890504 642122 notify.go:220] Checking for updates...
I0327 19:14:25.895531 642122 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18517-439728/kubeconfig
I0327 19:14:25.897163 642122 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18517-439728/.minikube
I0327 19:14:25.898605 642122 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0327 19:14:25.900202 642122 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0327 19:14:25.902694 642122 config.go:182] Loaded profile config "old-k8s-version-152774": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0327 19:14:25.905033 642122 out.go:177] * Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
I0327 19:14:25.906600 642122 driver.go:392] Setting default libvirt URI to qemu:///system
I0327 19:14:25.926219 642122 docker.go:122] docker version: linux-26.0.0:Docker Engine - Community
I0327 19:14:25.926347 642122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0327 19:14:25.990712 642122 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:54 OomKillDisable:true NGoroutines:67 SystemTime:2024-03-27 19:14:25.979199995 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0327 19:14:25.990839 642122 docker.go:295] overlay module found
I0327 19:14:25.998527 642122 out.go:177] * Using the docker driver based on existing profile
I0327 19:14:26.000696 642122 start.go:297] selected driver: docker
I0327 19:14:26.000719 642122 start.go:901] validating driver "docker" against &{Name:old-k8s-version-152774 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-152774 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0327 19:14:26.000871 642122 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0327 19:14:26.001595 642122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0327 19:14:26.077472 642122 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:54 OomKillDisable:true NGoroutines:67 SystemTime:2024-03-27 19:14:26.068316427 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0327 19:14:26.077822 642122 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0327 19:14:26.077881 642122 cni.go:84] Creating CNI manager for ""
I0327 19:14:26.077895 642122 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0327 19:14:26.077936 642122 start.go:340] cluster config:
{Name:old-k8s-version-152774 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-152774 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime
:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: Mou
ntMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0327 19:14:26.080326 642122 out.go:177] * Starting "old-k8s-version-152774" primary control-plane node in "old-k8s-version-152774" cluster
I0327 19:14:26.082155 642122 cache.go:121] Beginning downloading kic base image for docker with containerd
I0327 19:14:26.084199 642122 out.go:177] * Pulling base image v0.0.43-beta.0 ...
I0327 19:14:26.086243 642122 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0327 19:14:26.086303 642122 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0327 19:14:26.086317 642122 cache.go:56] Caching tarball of preloaded images
I0327 19:14:26.086326 642122 image.go:79] Checking for gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 in local docker daemon
I0327 19:14:26.086464 642122 preload.go:173] Found /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0327 19:14:26.086476 642122 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0327 19:14:26.086636 642122 profile.go:142] Saving config to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/config.json ...
I0327 19:14:26.101198 642122 image.go:83] Found gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 in local docker daemon, skipping pull
I0327 19:14:26.101222 642122 cache.go:144] gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 exists in daemon, skipping load
I0327 19:14:26.101246 642122 cache.go:194] Successfully downloaded all kic artifacts
I0327 19:14:26.101274 642122 start.go:360] acquireMachinesLock for old-k8s-version-152774: {Name:mkb2dc4a6b785e04ab44f61c54ed704a68357a2f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0327 19:14:26.101354 642122 start.go:364] duration metric: took 58.057µs to acquireMachinesLock for "old-k8s-version-152774"
I0327 19:14:26.101482 642122 start.go:96] Skipping create...Using existing machine configuration
I0327 19:14:26.101493 642122 fix.go:54] fixHost starting:
I0327 19:14:26.101780 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:26.117506 642122 fix.go:112] recreateIfNeeded on old-k8s-version-152774: state=Stopped err=<nil>
W0327 19:14:26.117539 642122 fix.go:138] unexpected machine state, will restart: <nil>
I0327 19:14:26.119789 642122 out.go:177] * Restarting existing docker container for "old-k8s-version-152774" ...
I0327 19:14:26.121743 642122 cli_runner.go:164] Run: docker start old-k8s-version-152774
I0327 19:14:26.474382 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:26.492266 642122 kic.go:430] container "old-k8s-version-152774" state is running.
I0327 19:14:26.492655 642122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-152774
I0327 19:14:26.508451 642122 profile.go:142] Saving config to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/config.json ...
I0327 19:14:26.508674 642122 machine.go:94] provisionDockerMachine start ...
I0327 19:14:26.508735 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:26.524677 642122 main.go:141] libmachine: Using SSH client type: native
I0327 19:14:26.524960 642122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33462 <nil> <nil>}
I0327 19:14:26.524976 642122 main.go:141] libmachine: About to run SSH command:
hostname
I0327 19:14:26.527192 642122 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0327 19:14:29.653891 642122 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-152774
I0327 19:14:29.653915 642122 ubuntu.go:169] provisioning hostname "old-k8s-version-152774"
I0327 19:14:29.653978 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:29.670519 642122 main.go:141] libmachine: Using SSH client type: native
I0327 19:14:29.670757 642122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33462 <nil> <nil>}
I0327 19:14:29.670775 642122 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-152774 && echo "old-k8s-version-152774" | sudo tee /etc/hostname
I0327 19:14:29.810489 642122 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-152774
I0327 19:14:29.810584 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:29.830320 642122 main.go:141] libmachine: Using SSH client type: native
I0327 19:14:29.830644 642122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33462 <nil> <nil>}
I0327 19:14:29.830664 642122 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-152774' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-152774/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-152774' | sudo tee -a /etc/hosts;
fi
fi
I0327 19:14:29.958652 642122 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0327 19:14:29.958692 642122 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18517-439728/.minikube CaCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18517-439728/.minikube}
I0327 19:14:29.958715 642122 ubuntu.go:177] setting up certificates
I0327 19:14:29.958725 642122 provision.go:84] configureAuth start
I0327 19:14:29.958787 642122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-152774
I0327 19:14:29.974786 642122 provision.go:143] copyHostCerts
I0327 19:14:29.974868 642122 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem, removing ...
I0327 19:14:29.974892 642122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem
I0327 19:14:29.974979 642122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem (1082 bytes)
I0327 19:14:29.975104 642122 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem, removing ...
I0327 19:14:29.975114 642122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem
I0327 19:14:29.975147 642122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem (1123 bytes)
I0327 19:14:29.975213 642122 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem, removing ...
I0327 19:14:29.975234 642122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem
I0327 19:14:29.975262 642122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem (1675 bytes)
I0327 19:14:29.975334 642122 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-152774 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-152774]
I0327 19:14:30.732565 642122 provision.go:177] copyRemoteCerts
I0327 19:14:30.732655 642122 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0327 19:14:30.732706 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:30.758551 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:30.862174 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0327 19:14:30.905467 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0327 19:14:30.936068 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0327 19:14:30.964253 642122 provision.go:87] duration metric: took 1.005514513s to configureAuth
I0327 19:14:30.964283 642122 ubuntu.go:193] setting minikube options for container-runtime
I0327 19:14:30.964532 642122 config.go:182] Loaded profile config "old-k8s-version-152774": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0327 19:14:30.964548 642122 machine.go:97] duration metric: took 4.455862533s to provisionDockerMachine
I0327 19:14:30.964577 642122 start.go:293] postStartSetup for "old-k8s-version-152774" (driver="docker")
I0327 19:14:30.964590 642122 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0327 19:14:30.964657 642122 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0327 19:14:30.964729 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:30.983590 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:31.081354 642122 ssh_runner.go:195] Run: cat /etc/os-release
I0327 19:14:31.085235 642122 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0327 19:14:31.085283 642122 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0327 19:14:31.085299 642122 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0327 19:14:31.085307 642122 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0327 19:14:31.085317 642122 filesync.go:126] Scanning /home/jenkins/minikube-integration/18517-439728/.minikube/addons for local assets ...
I0327 19:14:31.085374 642122 filesync.go:126] Scanning /home/jenkins/minikube-integration/18517-439728/.minikube/files for local assets ...
I0327 19:14:31.085454 642122 filesync.go:149] local asset: /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem -> 4451462.pem in /etc/ssl/certs
I0327 19:14:31.085562 642122 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0327 19:14:31.096461 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem --> /etc/ssl/certs/4451462.pem (1708 bytes)
I0327 19:14:31.123993 642122 start.go:296] duration metric: took 159.398021ms for postStartSetup
I0327 19:14:31.124107 642122 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0327 19:14:31.124154 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:31.144936 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:31.233166 642122 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0327 19:14:31.238941 642122 fix.go:56] duration metric: took 5.137443264s for fixHost
I0327 19:14:31.238976 642122 start.go:83] releasing machines lock for "old-k8s-version-152774", held for 5.137612472s
I0327 19:14:31.239059 642122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-152774
I0327 19:14:31.265996 642122 ssh_runner.go:195] Run: cat /version.json
I0327 19:14:31.266075 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:31.266340 642122 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0327 19:14:31.266395 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:31.297382 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:31.304716 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:31.394311 642122 ssh_runner.go:195] Run: systemctl --version
I0327 19:14:31.530709 642122 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0327 19:14:31.535175 642122 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0327 19:14:31.555505 642122 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0327 19:14:31.555626 642122 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0327 19:14:31.565513 642122 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0327 19:14:31.565591 642122 start.go:494] detecting cgroup driver to use...
I0327 19:14:31.565650 642122 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0327 19:14:31.565736 642122 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0327 19:14:31.590570 642122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0327 19:14:31.613815 642122 docker.go:217] disabling cri-docker service (if available) ...
I0327 19:14:31.613927 642122 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0327 19:14:31.643164 642122 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0327 19:14:31.659222 642122 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0327 19:14:31.791734 642122 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0327 19:14:31.944223 642122 docker.go:233] disabling docker service ...
I0327 19:14:31.944326 642122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0327 19:14:31.974220 642122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0327 19:14:31.990669 642122 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0327 19:14:32.189538 642122 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0327 19:14:32.340136 642122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0327 19:14:32.359082 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0327 19:14:32.383072 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0327 19:14:32.403696 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0327 19:14:32.419303 642122 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0327 19:14:32.419382 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0327 19:14:32.432018 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0327 19:14:32.447965 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0327 19:14:32.457973 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0327 19:14:32.473911 642122 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0327 19:14:32.486199 642122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0327 19:14:32.501649 642122 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0327 19:14:32.513319 642122 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0327 19:14:32.527139 642122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0327 19:14:32.701048 642122 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0327 19:14:33.005141 642122 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0327 19:14:33.005348 642122 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0327 19:14:33.018768 642122 start.go:562] Will wait 60s for crictl version
I0327 19:14:33.018896 642122 ssh_runner.go:195] Run: which crictl
I0327 19:14:33.023035 642122 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0327 19:14:33.091524 642122 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0327 19:14:33.091600 642122 ssh_runner.go:195] Run: containerd --version
I0327 19:14:33.124239 642122 ssh_runner.go:195] Run: containerd --version
I0327 19:14:33.166315 642122 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
I0327 19:14:33.168504 642122 cli_runner.go:164] Run: docker network inspect old-k8s-version-152774 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0327 19:14:33.184849 642122 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0327 19:14:33.189036 642122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0327 19:14:33.213645 642122 kubeadm.go:877] updating cluster {Name:old-k8s-version-152774 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-152774 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNa
mes:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/min
ikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0327 19:14:33.213770 642122 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0327 19:14:33.213833 642122 ssh_runner.go:195] Run: sudo crictl images --output json
I0327 19:14:33.284805 642122 containerd.go:627] all images are preloaded for containerd runtime.
I0327 19:14:33.284829 642122 containerd.go:534] Images already preloaded, skipping extraction
I0327 19:14:33.284887 642122 ssh_runner.go:195] Run: sudo crictl images --output json
I0327 19:14:33.389147 642122 containerd.go:627] all images are preloaded for containerd runtime.
I0327 19:14:33.389170 642122 cache_images.go:84] Images are preloaded, skipping loading
I0327 19:14:33.389178 642122 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.20.0 containerd true true} ...
I0327 19:14:33.389320 642122 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-152774 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-152774 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0327 19:14:33.389398 642122 ssh_runner.go:195] Run: sudo crictl info
I0327 19:14:33.478875 642122 cni.go:84] Creating CNI manager for ""
I0327 19:14:33.478900 642122 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0327 19:14:33.478971 642122 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0327 19:14:33.478999 642122 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-152774 NodeName:old-k8s-version-152774 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0327 19:14:33.479141 642122 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-152774"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0327 19:14:33.479226 642122 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0327 19:14:33.492379 642122 binaries.go:44] Found k8s binaries, skipping transfer
I0327 19:14:33.492455 642122 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0327 19:14:33.516110 642122 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0327 19:14:33.552703 642122 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0327 19:14:33.585819 642122 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0327 19:14:33.613920 642122 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0327 19:14:33.621950 642122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0327 19:14:33.640391 642122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0327 19:14:33.770927 642122 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0327 19:14:33.791237 642122 certs.go:68] Setting up /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774 for IP: 192.168.85.2
I0327 19:14:33.791312 642122 certs.go:194] generating shared ca certs ...
I0327 19:14:33.791357 642122 certs.go:226] acquiring lock for ca certs: {Name:mke441edb53252dfa332e7961e8e7663942b248d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:14:33.791535 642122 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18517-439728/.minikube/ca.key
I0327 19:14:33.791616 642122 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.key
I0327 19:14:33.791652 642122 certs.go:256] generating profile certs ...
I0327 19:14:33.791784 642122 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/client.key
I0327 19:14:33.791905 642122 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/apiserver.key.86ad8dd0
I0327 19:14:33.791989 642122 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/proxy-client.key
I0327 19:14:33.792152 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146.pem (1338 bytes)
W0327 19:14:33.792221 642122 certs.go:480] ignoring /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146_empty.pem, impossibly tiny 0 bytes
I0327 19:14:33.792249 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem (1679 bytes)
I0327 19:14:33.792313 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem (1082 bytes)
I0327 19:14:33.792373 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem (1123 bytes)
I0327 19:14:33.792424 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem (1675 bytes)
I0327 19:14:33.792499 642122 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem (1708 bytes)
I0327 19:14:33.793375 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0327 19:14:33.834620 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0327 19:14:33.882569 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0327 19:14:33.936877 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0327 19:14:33.984725 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0327 19:14:34.072851 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0327 19:14:34.114357 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0327 19:14:34.149959 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/old-k8s-version-152774/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0327 19:14:34.192588 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0327 19:14:34.227406 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146.pem --> /usr/share/ca-certificates/445146.pem (1338 bytes)
I0327 19:14:34.258926 642122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem --> /usr/share/ca-certificates/4451462.pem (1708 bytes)
I0327 19:14:34.291042 642122 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0327 19:14:34.314232 642122 ssh_runner.go:195] Run: openssl version
I0327 19:14:34.324475 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0327 19:14:34.338071 642122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0327 19:14:34.344678 642122 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 27 18:28 /usr/share/ca-certificates/minikubeCA.pem
I0327 19:14:34.344744 642122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0327 19:14:34.353056 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0327 19:14:34.363551 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/445146.pem && ln -fs /usr/share/ca-certificates/445146.pem /etc/ssl/certs/445146.pem"
I0327 19:14:34.380650 642122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/445146.pem
I0327 19:14:34.386155 642122 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 27 18:34 /usr/share/ca-certificates/445146.pem
I0327 19:14:34.386227 642122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/445146.pem
I0327 19:14:34.394565 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/445146.pem /etc/ssl/certs/51391683.0"
I0327 19:14:34.404703 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4451462.pem && ln -fs /usr/share/ca-certificates/4451462.pem /etc/ssl/certs/4451462.pem"
I0327 19:14:34.423109 642122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4451462.pem
I0327 19:14:34.426876 642122 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 27 18:34 /usr/share/ca-certificates/4451462.pem
I0327 19:14:34.426986 642122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4451462.pem
I0327 19:14:34.435714 642122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4451462.pem /etc/ssl/certs/3ec20f2e.0"
I0327 19:14:34.452732 642122 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0327 19:14:34.457963 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0327 19:14:34.469180 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0327 19:14:34.478240 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0327 19:14:34.487599 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0327 19:14:34.497565 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0327 19:14:34.506849 642122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0327 19:14:34.514625 642122 kubeadm.go:391] StartCluster: {Name:old-k8s-version-152774 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-152774 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames
:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/miniku
be-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0327 19:14:34.514739 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0327 19:14:34.514798 642122 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0327 19:14:34.568268 642122 cri.go:89] found id: "1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:14:34.568293 642122 cri.go:89] found id: "748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:14:34.568298 642122 cri.go:89] found id: "63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:14:34.568301 642122 cri.go:89] found id: "2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:14:34.568305 642122 cri.go:89] found id: "35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:14:34.568310 642122 cri.go:89] found id: "1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:14:34.568313 642122 cri.go:89] found id: "2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:14:34.568316 642122 cri.go:89] found id: "2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:14:34.568319 642122 cri.go:89] found id: ""
I0327 19:14:34.568385 642122 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0327 19:14:34.588444 642122 cri.go:116] JSON = null
W0327 19:14:34.588491 642122 kubeadm.go:398] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0327 19:14:34.588566 642122 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0327 19:14:34.601296 642122 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0327 19:14:34.601318 642122 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0327 19:14:34.601336 642122 kubeadm.go:587] restartPrimaryControlPlane start ...
I0327 19:14:34.601383 642122 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0327 19:14:34.610889 642122 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0327 19:14:34.611462 642122 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-152774" does not appear in /home/jenkins/minikube-integration/18517-439728/kubeconfig
I0327 19:14:34.611760 642122 kubeconfig.go:62] /home/jenkins/minikube-integration/18517-439728/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-152774" cluster setting kubeconfig missing "old-k8s-version-152774" context setting]
I0327 19:14:34.612269 642122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/kubeconfig: {Name:mk2fd9b8c6944ee7a88d0c9ed4d76f33c13b84e8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:14:34.613552 642122 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0327 19:14:34.626716 642122 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.85.2
I0327 19:14:34.626751 642122 kubeadm.go:591] duration metric: took 25.408175ms to restartPrimaryControlPlane
I0327 19:14:34.626760 642122 kubeadm.go:393] duration metric: took 112.143832ms to StartCluster
I0327 19:14:34.626784 642122 settings.go:142] acquiring lock: {Name:mk721e9764b3eb47bed1c7c2ad6b96c8ecb798de Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:14:34.626841 642122 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18517-439728/kubeconfig
I0327 19:14:34.627828 642122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/kubeconfig: {Name:mk2fd9b8c6944ee7a88d0c9ed4d76f33c13b84e8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:14:34.628035 642122 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0327 19:14:34.633510 642122 out.go:177] * Verifying Kubernetes components...
I0327 19:14:34.628407 642122 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0327 19:14:34.628519 642122 config.go:182] Loaded profile config "old-k8s-version-152774": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0327 19:14:34.635567 642122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0327 19:14:34.635680 642122 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-152774"
I0327 19:14:34.635707 642122 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-152774"
W0327 19:14:34.635725 642122 addons.go:243] addon storage-provisioner should already be in state true
I0327 19:14:34.635752 642122 host.go:66] Checking if "old-k8s-version-152774" exists ...
I0327 19:14:34.636226 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:34.636399 642122 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-152774"
I0327 19:14:34.636430 642122 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-152774"
I0327 19:14:34.636701 642122 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-152774"
I0327 19:14:34.636722 642122 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-152774"
W0327 19:14:34.636728 642122 addons.go:243] addon metrics-server should already be in state true
I0327 19:14:34.636802 642122 host.go:66] Checking if "old-k8s-version-152774" exists ...
I0327 19:14:34.637167 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:34.637412 642122 addons.go:69] Setting dashboard=true in profile "old-k8s-version-152774"
I0327 19:14:34.637454 642122 addons.go:234] Setting addon dashboard=true in "old-k8s-version-152774"
W0327 19:14:34.637476 642122 addons.go:243] addon dashboard should already be in state true
I0327 19:14:34.637511 642122 host.go:66] Checking if "old-k8s-version-152774" exists ...
I0327 19:14:34.637901 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:34.638296 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:34.712824 642122 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0327 19:14:34.715873 642122 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0327 19:14:34.721094 642122 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0327 19:14:34.721120 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0327 19:14:34.721195 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:34.722401 642122 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-152774"
W0327 19:14:34.722434 642122 addons.go:243] addon default-storageclass should already be in state true
I0327 19:14:34.722459 642122 host.go:66] Checking if "old-k8s-version-152774" exists ...
I0327 19:14:34.722854 642122 cli_runner.go:164] Run: docker container inspect old-k8s-version-152774 --format={{.State.Status}}
I0327 19:14:34.716047 642122 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0327 19:14:34.723321 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0327 19:14:34.723371 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:34.725547 642122 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0327 19:14:34.729266 642122 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0327 19:14:34.731382 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0327 19:14:34.731405 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0327 19:14:34.731469 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:34.777507 642122 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0327 19:14:34.777530 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0327 19:14:34.778580 642122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-152774
I0327 19:14:34.780071 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:34.792235 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:34.798439 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:34.810739 642122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33462 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/old-k8s-version-152774/id_rsa Username:docker}
I0327 19:14:34.861303 642122 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0327 19:14:34.898172 642122 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-152774" to be "Ready" ...
I0327 19:14:34.985801 642122 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0327 19:14:34.985872 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0327 19:14:34.993130 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0327 19:14:34.993204 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0327 19:14:35.039294 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0327 19:14:35.085433 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0327 19:14:35.085512 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0327 19:14:35.120023 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0327 19:14:35.120395 642122 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0327 19:14:35.120445 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0327 19:14:35.129737 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0327 19:14:35.129813 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0327 19:14:35.189556 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0327 19:14:35.189709 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0327 19:14:35.228220 642122 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0327 19:14:35.228297 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0327 19:14:35.295373 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0327 19:14:35.295459 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0327 19:14:35.320972 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.321054 642122 retry.go:31] will retry after 223.036644ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.328926 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0327 19:14:35.365360 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0327 19:14:35.365436 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0327 19:14:35.403085 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0327 19:14:35.403107 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
W0327 19:14:35.462222 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.462304 642122 retry.go:31] will retry after 204.522732ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.471479 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0327 19:14:35.471557 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0327 19:14:35.519282 642122 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0327 19:14:35.519354 642122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
W0327 19:14:35.541308 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.541390 642122 retry.go:31] will retry after 351.100729ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.544675 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0327 19:14:35.558749 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0327 19:14:35.667053 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0327 19:14:35.734995 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.735076 642122 retry.go:31] will retry after 496.063239ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:35.771593 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.771672 642122 retry.go:31] will retry after 309.914974ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:35.847873 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.847955 642122 retry.go:31] will retry after 210.155858ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.893477 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:35.976012 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:35.976057 642122 retry.go:31] will retry after 287.922628ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.058365 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0327 19:14:36.081927 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0327 19:14:36.179618 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.179715 642122 retry.go:31] will retry after 839.457391ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:36.217162 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.217247 642122 retry.go:31] will retry after 382.773391ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.232346 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0327 19:14:36.264748 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:36.335677 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.335713 642122 retry.go:31] will retry after 337.026918ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:36.350023 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.350073 642122 retry.go:31] will retry after 427.211842ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.601099 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0327 19:14:36.673095 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0327 19:14:36.673277 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.673301 642122 retry.go:31] will retry after 441.964435ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:36.763365 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.763408 642122 retry.go:31] will retry after 811.637701ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.777494 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:36.863621 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.863653 642122 retry.go:31] will retry after 614.825065ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:36.899106 642122 node_ready.go:53] error getting node "old-k8s-version-152774": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-152774": dial tcp 192.168.85.2:8443: connect: connection refused
I0327 19:14:37.019414 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0327 19:14:37.108037 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.108070 642122 retry.go:31] will retry after 425.228149ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.116211 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0327 19:14:37.186900 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.186944 642122 retry.go:31] will retry after 842.420326ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.478943 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0327 19:14:37.534367 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0327 19:14:37.569959 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.569990 642122 retry.go:31] will retry after 1.758119243s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.575229 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0327 19:14:37.651027 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.651055 642122 retry.go:31] will retry after 1.30312997s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:37.680751 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:37.680782 642122 retry.go:31] will retry after 968.999206ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.029841 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0327 19:14:38.110818 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.110854 642122 retry.go:31] will retry after 718.778468ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.649958 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0327 19:14:38.736242 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.736274 642122 retry.go:31] will retry after 1.083278938s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.830522 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0327 19:14:38.897069 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.897103 642122 retry.go:31] will retry after 1.587565208s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:38.899461 642122 node_ready.go:53] error getting node "old-k8s-version-152774": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-152774": dial tcp 192.168.85.2:8443: connect: connection refused
I0327 19:14:38.954692 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0327 19:14:39.036090 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:39.036125 642122 retry.go:31] will retry after 2.42494031s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:39.328495 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:39.400844 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:39.400886 642122 retry.go:31] will retry after 1.41763269s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:39.820420 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0327 19:14:39.898098 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:39.898127 642122 retry.go:31] will retry after 3.298572306s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:40.484843 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0327 19:14:40.573439 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:40.573471 642122 retry.go:31] will retry after 2.597219355s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:40.818807 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:40.910606 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:40.910645 642122 retry.go:31] will retry after 1.91831676s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:41.399585 642122 node_ready.go:53] error getting node "old-k8s-version-152774": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-152774": dial tcp 192.168.85.2:8443: connect: connection refused
I0327 19:14:41.461964 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0327 19:14:41.545039 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:41.545070 642122 retry.go:31] will retry after 3.731950773s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:42.829705 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0327 19:14:42.952448 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:42.952478 642122 retry.go:31] will retry after 3.660655724s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:43.170863 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0327 19:14:43.197307 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0327 19:14:43.446326 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:43.446399 642122 retry.go:31] will retry after 2.440151527s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0327 19:14:43.606990 642122 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:43.607025 642122 retry.go:31] will retry after 6.350369319s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0327 19:14:45.277762 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0327 19:14:45.887517 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0327 19:14:46.613340 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0327 19:14:49.957700 642122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0327 19:14:51.709801 642122 node_ready.go:49] node "old-k8s-version-152774" has status "Ready":"True"
I0327 19:14:51.709827 642122 node_ready.go:38] duration metric: took 16.811620598s for node "old-k8s-version-152774" to be "Ready" ...
I0327 19:14:51.709838 642122 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0327 19:14:51.940485 642122 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-p8r76" in "kube-system" namespace to be "Ready" ...
I0327 19:14:52.018588 642122 pod_ready.go:92] pod "coredns-74ff55c5b-p8r76" in "kube-system" namespace has status "Ready":"True"
I0327 19:14:52.018664 642122 pod_ready.go:81] duration metric: took 78.096837ms for pod "coredns-74ff55c5b-p8r76" in "kube-system" namespace to be "Ready" ...
I0327 19:14:52.018692 642122 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:14:52.054207 642122 pod_ready.go:92] pod "etcd-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"True"
I0327 19:14:52.054284 642122 pod_ready.go:81] duration metric: took 35.571521ms for pod "etcd-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:14:52.054322 642122 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:14:52.324900 642122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (7.047003937s)
I0327 19:14:53.203643 642122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.316072002s)
I0327 19:14:53.203734 642122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (6.590281374s)
I0327 19:14:53.203776 642122 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-152774"
I0327 19:14:53.206004 642122 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-152774 addons enable metrics-server
I0327 19:14:53.203845 642122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (3.246111467s)
I0327 19:14:53.207811 642122 out.go:177] * Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
I0327 19:14:53.209525 642122 addons.go:505] duration metric: took 18.581115472s for enable addons: enabled=[default-storageclass metrics-server storage-provisioner dashboard]
I0327 19:14:54.063997 642122 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:14:56.560228 642122 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:14:58.561030 642122 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"True"
I0327 19:14:58.561057 642122 pod_ready.go:81] duration metric: took 6.50669981s for pod "kube-apiserver-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:14:58.561070 642122 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:15:00.571978 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:03.069177 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:05.568074 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:08.068071 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:10.071207 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:12.567815 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:14.568121 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:16.570207 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:19.068975 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:21.072452 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:23.567974 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:25.568433 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:28.068890 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:30.095302 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:32.567789 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:35.067853 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:37.067950 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:39.074449 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:41.567656 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:43.567831 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:45.569675 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:48.068569 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:50.567445 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:52.567877 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:55.068261 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:57.068689 642122 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:15:57.568295 642122 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"True"
I0327 19:15:57.568380 642122 pod_ready.go:81] duration metric: took 59.007299358s for pod "kube-controller-manager-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:15:57.568408 642122 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-9tnq4" in "kube-system" namespace to be "Ready" ...
I0327 19:15:57.574673 642122 pod_ready.go:92] pod "kube-proxy-9tnq4" in "kube-system" namespace has status "Ready":"True"
I0327 19:15:57.574698 642122 pod_ready.go:81] duration metric: took 6.275418ms for pod "kube-proxy-9tnq4" in "kube-system" namespace to be "Ready" ...
I0327 19:15:57.574709 642122 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:15:59.581460 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:01.581552 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:04.080652 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:06.081673 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:08.585911 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:11.080979 642122 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:12.582102 642122 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace has status "Ready":"True"
I0327 19:16:12.582133 642122 pod_ready.go:81] duration metric: took 15.00741056s for pod "kube-scheduler-old-k8s-version-152774" in "kube-system" namespace to be "Ready" ...
I0327 19:16:12.582147 642122 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace to be "Ready" ...
I0327 19:16:14.587978 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:16.588979 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:19.088906 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:21.588292 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:23.588574 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:25.588921 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:28.088163 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:30.092004 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:32.591681 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:35.090222 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:37.588788 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:39.588913 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:42.109957 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:44.588075 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:46.588953 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:48.589342 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:51.089182 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:53.095481 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:55.589570 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:57.592323 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:16:59.594699 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:02.090450 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:04.589989 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:07.089244 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:09.587689 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:12.095283 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:14.589753 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:17.088974 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:19.589667 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:22.088491 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:24.089919 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:26.588066 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:29.088700 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:31.089261 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:33.096655 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:35.589664 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:38.089310 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:40.089427 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:42.090703 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:44.588367 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:46.589338 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:49.088781 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:51.089292 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:53.587481 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:55.588511 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:17:58.089864 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:00.167611 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:02.589539 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:04.589599 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:06.589937 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:09.094103 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:11.589226 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:14.089022 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:16.089314 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:18.089883 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:20.090781 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:22.589367 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:24.592942 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:27.091563 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:29.593534 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:32.089748 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:34.589337 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:36.589683 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:39.089296 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:41.093140 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:43.589076 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:46.089682 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:48.089910 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:50.090131 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:52.588599 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:54.588891 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:56.590081 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:18:59.089371 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:01.588663 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:03.686497 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:06.089603 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:08.588461 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:10.589416 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:13.093791 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:15.589403 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:17.589495 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:19.589816 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:22.094205 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:24.588384 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:26.590682 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:29.088052 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:31.089679 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:33.099672 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:35.590186 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:38.088959 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:40.089308 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:42.093928 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:44.097446 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:46.589820 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:48.590131 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:51.091287 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:53.592872 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:56.091060 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:58.589083 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:00.617404 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:03.090739 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:05.119152 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:07.588744 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:09.590292 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:12.091204 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:12.591512 642122 pod_ready.go:81] duration metric: took 4m0.009346666s for pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace to be "Ready" ...
E0327 19:20:12.591546 642122 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0327 19:20:12.591555 642122 pod_ready.go:38] duration metric: took 5m20.881706224s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0327 19:20:12.591573 642122 api_server.go:52] waiting for apiserver process to appear ...
I0327 19:20:12.591611 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0327 19:20:12.591681 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0327 19:20:12.661305 642122 cri.go:89] found id: "0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:12.661383 642122 cri.go:89] found id: "2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:12.661391 642122 cri.go:89] found id: ""
I0327 19:20:12.661399 642122 logs.go:276] 2 containers: [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c]
I0327 19:20:12.661490 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.665803 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.669847 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0327 19:20:12.669969 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0327 19:20:12.722052 642122 cri.go:89] found id: "52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:12.722121 642122 cri.go:89] found id: "2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:12.722139 642122 cri.go:89] found id: ""
I0327 19:20:12.722162 642122 logs.go:276] 2 containers: [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42]
I0327 19:20:12.722248 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.726345 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.730599 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0327 19:20:12.730733 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0327 19:20:12.805110 642122 cri.go:89] found id: "6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:12.805182 642122 cri.go:89] found id: "1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:12.805200 642122 cri.go:89] found id: ""
I0327 19:20:12.805225 642122 logs.go:276] 2 containers: [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b]
I0327 19:20:12.805323 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.809380 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.814289 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0327 19:20:12.814427 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0327 19:20:12.864113 642122 cri.go:89] found id: "45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:12.864193 642122 cri.go:89] found id: "35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:12.864212 642122 cri.go:89] found id: ""
I0327 19:20:12.864235 642122 logs.go:276] 2 containers: [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424]
I0327 19:20:12.864322 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.868917 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.873038 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0327 19:20:12.873178 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0327 19:20:12.923144 642122 cri.go:89] found id: "de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:12.923200 642122 cri.go:89] found id: "2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:12.923218 642122 cri.go:89] found id: ""
I0327 19:20:12.923240 642122 logs.go:276] 2 containers: [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83]
I0327 19:20:12.923324 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.927187 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.931355 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0327 19:20:12.931473 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0327 19:20:13.003546 642122 cri.go:89] found id: "6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:13.003627 642122 cri.go:89] found id: "1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:13.003646 642122 cri.go:89] found id: ""
I0327 19:20:13.003671 642122 logs.go:276] 2 containers: [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b]
I0327 19:20:13.003771 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.008616 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.018497 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0327 19:20:13.018630 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0327 19:20:13.085416 642122 cri.go:89] found id: "207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:13.085485 642122 cri.go:89] found id: "63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:13.085502 642122 cri.go:89] found id: ""
I0327 19:20:13.085525 642122 logs.go:276] 2 containers: [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737]
I0327 19:20:13.085611 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.090248 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.094274 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0327 19:20:13.094442 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0327 19:20:13.152485 642122 cri.go:89] found id: "63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:13.152547 642122 cri.go:89] found id: "748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:13.152578 642122 cri.go:89] found id: ""
I0327 19:20:13.152602 642122 logs.go:276] 2 containers: [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448]
I0327 19:20:13.152688 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.156834 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.160902 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0327 19:20:13.161040 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0327 19:20:13.213344 642122 cri.go:89] found id: "44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:13.213406 642122 cri.go:89] found id: ""
I0327 19:20:13.213439 642122 logs.go:276] 1 containers: [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585]
I0327 19:20:13.213526 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.221951 642122 logs.go:123] Gathering logs for etcd [2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42] ...
I0327 19:20:13.222022 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:13.297783 642122 logs.go:123] Gathering logs for storage-provisioner [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07] ...
I0327 19:20:13.297859 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:13.348878 642122 logs.go:123] Gathering logs for storage-provisioner [748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448] ...
I0327 19:20:13.348955 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:13.404969 642122 logs.go:123] Gathering logs for containerd ...
I0327 19:20:13.405044 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0327 19:20:13.465407 642122 logs.go:123] Gathering logs for container status ...
I0327 19:20:13.465484 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0327 19:20:13.541221 642122 logs.go:123] Gathering logs for coredns [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228] ...
I0327 19:20:13.541381 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:13.587393 642122 logs.go:123] Gathering logs for kube-scheduler [35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424] ...
I0327 19:20:13.587459 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:13.645192 642122 logs.go:123] Gathering logs for kubernetes-dashboard [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585] ...
I0327 19:20:13.645362 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:13.701791 642122 logs.go:123] Gathering logs for dmesg ...
I0327 19:20:13.701857 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0327 19:20:13.722862 642122 logs.go:123] Gathering logs for kube-apiserver [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68] ...
I0327 19:20:13.722982 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:13.814163 642122 logs.go:123] Gathering logs for kube-apiserver [2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c] ...
I0327 19:20:13.814197 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:13.876593 642122 logs.go:123] Gathering logs for etcd [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb] ...
I0327 19:20:13.876627 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:13.984118 642122 logs.go:123] Gathering logs for kube-scheduler [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e] ...
I0327 19:20:13.984151 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:14.056761 642122 logs.go:123] Gathering logs for kube-proxy [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5] ...
I0327 19:20:14.056833 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:14.112348 642122 logs.go:123] Gathering logs for kindnet [63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737] ...
I0327 19:20:14.112379 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:14.164125 642122 logs.go:123] Gathering logs for kubelet ...
I0327 19:20:14.164159 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0327 19:20:14.226581 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.640363 662 reflector.go:138] object-"kube-system"/"metrics-server-token-bntmf": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-bntmf" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227269 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686154 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8c799": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8c799" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227614 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686269 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-qwbnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-qwbnq" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227851 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686334 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228080 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686386 662 reflector.go:138] object-"default"/"default-token-l225m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-l225m" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228300 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688291 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228526 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688368 662 reflector.go:138] object-"kube-system"/"coredns-token-h758g": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-h758g" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228754 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688486 662 reflector.go:138] object-"kube-system"/"kindnet-token-pvkpn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pvkpn" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.236452 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.420152 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.236691 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.612018 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.241434 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:04 old-k8s-version-152774 kubelet[662]: E0327 19:15:04.457953 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.243155 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:15 old-k8s-version-152774 kubelet[662]: E0327 19:15:15.468690 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.243759 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:21 old-k8s-version-152774 kubelet[662]: E0327 19:15:21.722972 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.244105 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:22 old-k8s-version-152774 kubelet[662]: E0327 19:15:22.735339 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.244815 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:25 old-k8s-version-152774 kubelet[662]: E0327 19:15:25.475266 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.247381 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:27 old-k8s-version-152774 kubelet[662]: E0327 19:15:27.470008 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.248331 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:37 old-k8s-version-152774 kubelet[662]: E0327 19:15:37.769827 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.248554 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:40 old-k8s-version-152774 kubelet[662]: E0327 19:15:40.468375 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.248901 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:45 old-k8s-version-152774 kubelet[662]: E0327 19:15:45.475591 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.249104 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:55 old-k8s-version-152774 kubelet[662]: E0327 19:15:55.447878 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.249452 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:56 old-k8s-version-152774 kubelet[662]: E0327 19:15:56.447736 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.252060 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.478662 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.252409 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.843057 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.252903 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:15 old-k8s-version-152774 kubelet[662]: E0327 19:16:15.475201 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.253116 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:23 old-k8s-version-152774 kubelet[662]: E0327 19:16:23.455765 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.253468 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:26 old-k8s-version-152774 kubelet[662]: E0327 19:16:26.447884 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.253672 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:34 old-k8s-version-152774 kubelet[662]: E0327 19:16:34.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.254015 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:41 old-k8s-version-152774 kubelet[662]: E0327 19:16:41.447929 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.254220 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:45 old-k8s-version-152774 kubelet[662]: E0327 19:16:45.448175 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.254830 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:55 old-k8s-version-152774 kubelet[662]: E0327 19:16:55.932646 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.255039 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:59 old-k8s-version-152774 kubelet[662]: E0327 19:16:59.447909 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.255382 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:05 old-k8s-version-152774 kubelet[662]: E0327 19:17:05.475157 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.255584 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:14 old-k8s-version-152774 kubelet[662]: E0327 19:17:14.447954 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.256002 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:17 old-k8s-version-152774 kubelet[662]: E0327 19:17:17.447555 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.256209 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:25 old-k8s-version-152774 kubelet[662]: E0327 19:17:25.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.256552 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:28 old-k8s-version-152774 kubelet[662]: E0327 19:17:28.447656 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.259030 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:40 old-k8s-version-152774 kubelet[662]: E0327 19:17:40.456218 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.259390 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:41 old-k8s-version-152774 kubelet[662]: E0327 19:17:41.447800 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.259602 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:53 old-k8s-version-152774 kubelet[662]: E0327 19:17:53.447859 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.259960 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:56 old-k8s-version-152774 kubelet[662]: E0327 19:17:56.447992 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.260161 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:04 old-k8s-version-152774 kubelet[662]: E0327 19:18:04.448632 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.260509 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:09 old-k8s-version-152774 kubelet[662]: E0327 19:18:09.447980 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.260710 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:16 old-k8s-version-152774 kubelet[662]: E0327 19:18:16.448113 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.261319 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:25 old-k8s-version-152774 kubelet[662]: E0327 19:18:25.174857 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.261668 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:26 old-k8s-version-152774 kubelet[662]: E0327 19:18:26.177914 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.261869 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:29 old-k8s-version-152774 kubelet[662]: E0327 19:18:29.447914 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.262211 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:41 old-k8s-version-152774 kubelet[662]: E0327 19:18:41.447526 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.262524 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:42 old-k8s-version-152774 kubelet[662]: E0327 19:18:42.449330 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.262734 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:54 old-k8s-version-152774 kubelet[662]: E0327 19:18:54.448024 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.263080 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:55 old-k8s-version-152774 kubelet[662]: E0327 19:18:55.447433 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.263284 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:08 old-k8s-version-152774 kubelet[662]: E0327 19:19:08.447831 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.263712 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: E0327 19:19:09.447537 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.263915 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:20 old-k8s-version-152774 kubelet[662]: E0327 19:19:20.448039 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.264260 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: E0327 19:19:21.447551 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.264602 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: E0327 19:19:32.448689 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.264805 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265147 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.265365 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265579 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265930 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:14.265946 642122 logs.go:123] Gathering logs for describe nodes ...
I0327 19:20:14.265976 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0327 19:20:14.514548 642122 logs.go:123] Gathering logs for coredns [1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b] ...
I0327 19:20:14.514596 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:14.573317 642122 logs.go:123] Gathering logs for kube-proxy [2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83] ...
I0327 19:20:14.573352 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:14.624315 642122 logs.go:123] Gathering logs for kube-controller-manager [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f] ...
I0327 19:20:14.624344 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:14.717094 642122 logs.go:123] Gathering logs for kube-controller-manager [1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b] ...
I0327 19:20:14.717132 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:14.817864 642122 logs.go:123] Gathering logs for kindnet [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910] ...
I0327 19:20:14.817903 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:14.875620 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:14.875648 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0327 19:20:14.875726 642122 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0327 19:20:14.875757 642122 out.go:239] Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875791 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.875807 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875846 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875864 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:14.875877 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:14.875884 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:20:24.877342 642122 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0327 19:20:24.891922 642122 api_server.go:72] duration metric: took 5m50.263854092s to wait for apiserver process to appear ...
I0327 19:20:24.891946 642122 api_server.go:88] waiting for apiserver healthz status ...
I0327 19:20:24.891980 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0327 19:20:24.892035 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0327 19:20:24.974524 642122 cri.go:89] found id: "0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:24.974547 642122 cri.go:89] found id: "2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:24.974552 642122 cri.go:89] found id: ""
I0327 19:20:24.974560 642122 logs.go:276] 2 containers: [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c]
I0327 19:20:24.974621 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:24.980100 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:24.984691 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0327 19:20:24.984765 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0327 19:20:25.098328 642122 cri.go:89] found id: "52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:25.098353 642122 cri.go:89] found id: "2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:25.098358 642122 cri.go:89] found id: ""
I0327 19:20:25.098366 642122 logs.go:276] 2 containers: [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42]
I0327 19:20:25.098445 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.103699 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.110237 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0327 19:20:25.110313 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0327 19:20:25.194586 642122 cri.go:89] found id: "6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:25.194623 642122 cri.go:89] found id: "1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:25.194628 642122 cri.go:89] found id: ""
I0327 19:20:25.194636 642122 logs.go:276] 2 containers: [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b]
I0327 19:20:25.194704 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.199613 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.203738 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0327 19:20:25.203822 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0327 19:20:25.268088 642122 cri.go:89] found id: "45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:25.268115 642122 cri.go:89] found id: "35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:25.268120 642122 cri.go:89] found id: ""
I0327 19:20:25.268128 642122 logs.go:276] 2 containers: [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424]
I0327 19:20:25.268186 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.274153 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.278971 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0327 19:20:25.279050 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0327 19:20:25.386946 642122 cri.go:89] found id: "de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:25.386971 642122 cri.go:89] found id: "2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:25.386977 642122 cri.go:89] found id: ""
I0327 19:20:25.386984 642122 logs.go:276] 2 containers: [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83]
I0327 19:20:25.387041 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.393856 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.403946 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0327 19:20:25.404036 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0327 19:20:25.455380 642122 cri.go:89] found id: "6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:25.455403 642122 cri.go:89] found id: "1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:25.455408 642122 cri.go:89] found id: ""
I0327 19:20:25.455416 642122 logs.go:276] 2 containers: [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b]
I0327 19:20:25.455487 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.460988 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.465435 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0327 19:20:25.465532 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0327 19:20:25.545617 642122 cri.go:89] found id: "207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:25.545642 642122 cri.go:89] found id: "63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:25.545648 642122 cri.go:89] found id: ""
I0327 19:20:25.545665 642122 logs.go:276] 2 containers: [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737]
I0327 19:20:25.545722 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.549385 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.556895 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0327 19:20:25.556967 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0327 19:20:25.650130 642122 cri.go:89] found id: "63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:25.650154 642122 cri.go:89] found id: "748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:25.650160 642122 cri.go:89] found id: ""
I0327 19:20:25.650168 642122 logs.go:276] 2 containers: [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448]
I0327 19:20:25.650225 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.654924 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.658948 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0327 19:20:25.659030 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0327 19:20:25.718309 642122 cri.go:89] found id: "44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:25.718331 642122 cri.go:89] found id: ""
I0327 19:20:25.718340 642122 logs.go:276] 1 containers: [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585]
I0327 19:20:25.718454 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.724763 642122 logs.go:123] Gathering logs for storage-provisioner [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07] ...
I0327 19:20:25.724792 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:25.771906 642122 logs.go:123] Gathering logs for container status ...
I0327 19:20:25.771943 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0327 19:20:25.826538 642122 logs.go:123] Gathering logs for kubelet ...
I0327 19:20:25.826571 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0327 19:20:25.895233 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.640363 662 reflector.go:138] object-"kube-system"/"metrics-server-token-bntmf": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-bntmf" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.895479 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686154 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8c799": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8c799" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.895697 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686269 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-qwbnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-qwbnq" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.900828 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686334 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901067 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686386 662 reflector.go:138] object-"default"/"default-token-l225m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-l225m" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901286 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688291 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901495 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688368 662 reflector.go:138] object-"kube-system"/"coredns-token-h758g": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-h758g" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901706 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688486 662 reflector.go:138] object-"kube-system"/"kindnet-token-pvkpn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pvkpn" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.909567 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.420152 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.909765 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.612018 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.918075 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:04 old-k8s-version-152774 kubelet[662]: E0327 19:15:04.457953 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.919807 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:15 old-k8s-version-152774 kubelet[662]: E0327 19:15:15.468690 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.920412 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:21 old-k8s-version-152774 kubelet[662]: E0327 19:15:21.722972 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.920750 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:22 old-k8s-version-152774 kubelet[662]: E0327 19:15:22.735339 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.921410 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:25 old-k8s-version-152774 kubelet[662]: E0327 19:15:25.475266 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.923922 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:27 old-k8s-version-152774 kubelet[662]: E0327 19:15:27.470008 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.924881 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:37 old-k8s-version-152774 kubelet[662]: E0327 19:15:37.769827 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.925068 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:40 old-k8s-version-152774 kubelet[662]: E0327 19:15:40.468375 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.925395 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:45 old-k8s-version-152774 kubelet[662]: E0327 19:15:45.475591 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.925577 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:55 old-k8s-version-152774 kubelet[662]: E0327 19:15:55.447878 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.925900 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:56 old-k8s-version-152774 kubelet[662]: E0327 19:15:56.447736 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.928453 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.478662 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.928780 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.843057 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929232 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:15 old-k8s-version-152774 kubelet[662]: E0327 19:16:15.475201 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929421 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:23 old-k8s-version-152774 kubelet[662]: E0327 19:16:23.455765 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.929776 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:26 old-k8s-version-152774 kubelet[662]: E0327 19:16:26.447884 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929965 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:34 old-k8s-version-152774 kubelet[662]: E0327 19:16:34.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.930292 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:41 old-k8s-version-152774 kubelet[662]: E0327 19:16:41.447929 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.930479 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:45 old-k8s-version-152774 kubelet[662]: E0327 19:16:45.448175 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.931141 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:55 old-k8s-version-152774 kubelet[662]: E0327 19:16:55.932646 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.931328 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:59 old-k8s-version-152774 kubelet[662]: E0327 19:16:59.447909 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.931653 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:05 old-k8s-version-152774 kubelet[662]: E0327 19:17:05.475157 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.931835 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:14 old-k8s-version-152774 kubelet[662]: E0327 19:17:14.447954 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.932243 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:17 old-k8s-version-152774 kubelet[662]: E0327 19:17:17.447555 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.932432 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:25 old-k8s-version-152774 kubelet[662]: E0327 19:17:25.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.932763 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:28 old-k8s-version-152774 kubelet[662]: E0327 19:17:28.447656 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.935363 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:40 old-k8s-version-152774 kubelet[662]: E0327 19:17:40.456218 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.935700 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:41 old-k8s-version-152774 kubelet[662]: E0327 19:17:41.447800 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.935885 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:53 old-k8s-version-152774 kubelet[662]: E0327 19:17:53.447859 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.936209 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:56 old-k8s-version-152774 kubelet[662]: E0327 19:17:56.447992 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.936397 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:04 old-k8s-version-152774 kubelet[662]: E0327 19:18:04.448632 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.936720 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:09 old-k8s-version-152774 kubelet[662]: E0327 19:18:09.447980 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.936903 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:16 old-k8s-version-152774 kubelet[662]: E0327 19:18:16.448113 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.937518 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:25 old-k8s-version-152774 kubelet[662]: E0327 19:18:25.174857 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.937849 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:26 old-k8s-version-152774 kubelet[662]: E0327 19:18:26.177914 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.938100 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:29 old-k8s-version-152774 kubelet[662]: E0327 19:18:29.447914 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.938448 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:41 old-k8s-version-152774 kubelet[662]: E0327 19:18:41.447526 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.938648 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:42 old-k8s-version-152774 kubelet[662]: E0327 19:18:42.449330 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.938833 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:54 old-k8s-version-152774 kubelet[662]: E0327 19:18:54.448024 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.939160 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:55 old-k8s-version-152774 kubelet[662]: E0327 19:18:55.447433 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.939343 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:08 old-k8s-version-152774 kubelet[662]: E0327 19:19:08.447831 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.939667 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: E0327 19:19:09.447537 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.939848 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:20 old-k8s-version-152774 kubelet[662]: E0327 19:19:20.448039 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.940171 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: E0327 19:19:21.447551 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.940503 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: E0327 19:19:32.448689 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.940685 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941011 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.941193 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941379 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941701 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.942013 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.942207 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:25.942222 642122 logs.go:123] Gathering logs for dmesg ...
I0327 19:20:25.942236 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0327 19:20:25.979844 642122 logs.go:123] Gathering logs for etcd [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb] ...
I0327 19:20:25.979986 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:26.072884 642122 logs.go:123] Gathering logs for kube-controller-manager [1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b] ...
I0327 19:20:26.072923 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:26.172742 642122 logs.go:123] Gathering logs for kube-apiserver [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68] ...
I0327 19:20:26.172822 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:26.247706 642122 logs.go:123] Gathering logs for kube-apiserver [2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c] ...
I0327 19:20:26.247782 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:26.399253 642122 logs.go:123] Gathering logs for kube-proxy [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5] ...
I0327 19:20:26.399333 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:26.539830 642122 logs.go:123] Gathering logs for kindnet [63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737] ...
I0327 19:20:26.539856 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:26.631701 642122 logs.go:123] Gathering logs for kube-scheduler [35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424] ...
I0327 19:20:26.631776 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:26.731781 642122 logs.go:123] Gathering logs for kube-controller-manager [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f] ...
I0327 19:20:26.731977 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:26.859144 642122 logs.go:123] Gathering logs for describe nodes ...
I0327 19:20:26.859224 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0327 19:20:27.130798 642122 logs.go:123] Gathering logs for etcd [2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42] ...
I0327 19:20:27.130873 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:27.193000 642122 logs.go:123] Gathering logs for coredns [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228] ...
I0327 19:20:27.193110 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:27.246520 642122 logs.go:123] Gathering logs for coredns [1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b] ...
I0327 19:20:27.246600 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:27.295770 642122 logs.go:123] Gathering logs for kubernetes-dashboard [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585] ...
I0327 19:20:27.295842 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:27.365331 642122 logs.go:123] Gathering logs for containerd ...
I0327 19:20:27.365410 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0327 19:20:27.424989 642122 logs.go:123] Gathering logs for kube-scheduler [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e] ...
I0327 19:20:27.425063 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:27.475636 642122 logs.go:123] Gathering logs for kube-proxy [2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83] ...
I0327 19:20:27.475662 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:27.532399 642122 logs.go:123] Gathering logs for kindnet [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910] ...
I0327 19:20:27.532469 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:27.584156 642122 logs.go:123] Gathering logs for storage-provisioner [748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448] ...
I0327 19:20:27.584249 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:27.642669 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:27.642695 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0327 19:20:27.642759 642122 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0327 19:20:27.642775 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642784 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642795 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:27.642818 642122 out.go:239] Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642846 642122 out.go:239] Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:27.642861 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:27.642868 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:20:37.643979 642122 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0327 19:20:37.653515 642122 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0327 19:20:37.656458 642122 out.go:177]
W0327 19:20:37.658607 642122 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0327 19:20:37.658642 642122 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0327 19:20:37.658659 642122 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0327 19:20:37.658665 642122 out.go:239] *
*
W0327 19:20:37.659646 642122 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0327 19:20:37.662693 642122 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-152774 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-152774
helpers_test.go:235: (dbg) docker inspect old-k8s-version-152774:
-- stdout --
[
{
"Id": "7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81",
"Created": "2024-03-27T19:11:12.001268287Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 642310,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-03-27T19:14:26.466035064Z",
"FinishedAt": "2024-03-27T19:14:25.312790557Z"
},
"Image": "sha256:f9b5358e8c18dbe49e632154cad75e0968b2e103f621caff2c3ed996f4155861",
"ResolvConfPath": "/var/lib/docker/containers/7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81/hostname",
"HostsPath": "/var/lib/docker/containers/7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81/hosts",
"LogPath": "/var/lib/docker/containers/7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81/7937ccdac2ee78a5985cb012134291acb2d612ef5368b2d20c5eed33dc37fa81-json.log",
"Name": "/old-k8s-version-152774",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-152774:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-152774",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/131d03e78b606b0fa47b6424fbb5fa8f59a998dd5341757c44adcc1b9945fa13-init/diff:/var/lib/docker/overlay2/d3825c572fee2e042ff40f342d10ba017c14d5e3937a1169672fb3a94c9f53ab/diff",
"MergedDir": "/var/lib/docker/overlay2/131d03e78b606b0fa47b6424fbb5fa8f59a998dd5341757c44adcc1b9945fa13/merged",
"UpperDir": "/var/lib/docker/overlay2/131d03e78b606b0fa47b6424fbb5fa8f59a998dd5341757c44adcc1b9945fa13/diff",
"WorkDir": "/var/lib/docker/overlay2/131d03e78b606b0fa47b6424fbb5fa8f59a998dd5341757c44adcc1b9945fa13/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-152774",
"Source": "/var/lib/docker/volumes/old-k8s-version-152774/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-152774",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-152774",
"name.minikube.sigs.k8s.io": "old-k8s-version-152774",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "908f61cf253b526b8c41e280db8d2e6fa218a5c96cb0dc8d18cd08fec3a41fa9",
"SandboxKey": "/var/run/docker/netns/908f61cf253b",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33462"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33461"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33458"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33460"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33459"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-152774": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:55:02",
"NetworkID": "4bcf2b9eae2a9c17d2b5f917d8314fc4e3ba3343afb5a0a157163f88f62760fe",
"EndpointID": "28797cb56f81601a762e594efa46eac7e01ce20b48efdecf622c86ac00619b21",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-152774",
"7937ccdac2ee"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-152774 -n old-k8s-version-152774
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-152774 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-152774 logs -n 25: (2.116909939s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
| start | -p cert-expiration-928990 | cert-expiration-928990 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:09 UTC | 27 Mar 24 19:10 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-580546 | force-systemd-env-580546 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:10 UTC | 27 Mar 24 19:10 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-580546 | force-systemd-env-580546 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:10 UTC | 27 Mar 24 19:10 UTC |
| start | -p cert-options-054100 | cert-options-054100 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:10 UTC | 27 Mar 24 19:11 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-054100 ssh | cert-options-054100 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:11 UTC | 27 Mar 24 19:11 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-054100 -- sudo | cert-options-054100 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:11 UTC | 27 Mar 24 19:11 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-054100 | cert-options-054100 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:11 UTC | 27 Mar 24 19:11 UTC |
| start | -p old-k8s-version-152774 | old-k8s-version-152774 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:11 UTC | 27 Mar 24 19:14 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-928990 | cert-expiration-928990 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:13 UTC | 27 Mar 24 19:13 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-928990 | cert-expiration-928990 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:13 UTC | 27 Mar 24 19:13 UTC |
| start | -p no-preload-430373 --memory=2200 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:13 UTC | 27 Mar 24 19:14 UTC |
| | --alsologtostderr --wait=true | | | | | |
| | --preload=false --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.30.0-beta.0 | | | | | |
| addons | enable metrics-server -p old-k8s-version-152774 | old-k8s-version-152774 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | 27 Mar 24 19:14 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-152774 | old-k8s-version-152774 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | 27 Mar 24 19:14 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-152774 | old-k8s-version-152774 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | 27 Mar 24 19:14 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-152774 | old-k8s-version-152774 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | 27 Mar 24 19:14 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:14 UTC | 27 Mar 24 19:15 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:15 UTC | 27 Mar 24 19:15 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-430373 --memory=2200 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:15 UTC | 27 Mar 24 19:19 UTC |
| | --alsologtostderr --wait=true | | | | | |
| | --preload=false --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.30.0-beta.0 | | | | | |
| image | no-preload-430373 image list | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | 27 Mar 24 19:19 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | 27 Mar 24 19:19 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | 27 Mar 24 19:19 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | 27 Mar 24 19:19 UTC |
| delete | -p no-preload-430373 | no-preload-430373 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | 27 Mar 24 19:19 UTC |
| start | -p embed-certs-328500 | embed-certs-328500 | jenkins | v1.33.0-beta.0 | 27 Mar 24 19:19 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.29.3 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/03/27 19:19:57
Running on machine: ip-172-31-30-239
Binary: Built with gc go1.22.1 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0327 19:19:57.317521 651901 out.go:291] Setting OutFile to fd 1 ...
I0327 19:19:57.317713 651901 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:19:57.317738 651901 out.go:304] Setting ErrFile to fd 2...
I0327 19:19:57.317764 651901 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:19:57.318091 651901 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18517-439728/.minikube/bin
I0327 19:19:57.318636 651901 out.go:298] Setting JSON to false
I0327 19:19:57.319810 651901 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":10935,"bootTime":1711556262,"procs":217,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1056-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0327 19:19:57.319921 651901 start.go:139] virtualization:
I0327 19:19:57.322986 651901 out.go:177] * [embed-certs-328500] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0327 19:19:57.325770 651901 out.go:177] - MINIKUBE_LOCATION=18517
I0327 19:19:57.328028 651901 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0327 19:19:57.325852 651901 notify.go:220] Checking for updates...
I0327 19:19:57.330043 651901 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18517-439728/kubeconfig
I0327 19:19:57.332315 651901 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18517-439728/.minikube
I0327 19:19:57.334391 651901 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0327 19:19:57.336676 651901 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0327 19:19:57.340020 651901 config.go:182] Loaded profile config "old-k8s-version-152774": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0327 19:19:57.340187 651901 driver.go:392] Setting default libvirt URI to qemu:///system
I0327 19:19:57.359441 651901 docker.go:122] docker version: linux-26.0.0:Docker Engine - Community
I0327 19:19:57.359578 651901 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0327 19:19:57.430579 651901 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-03-27 19:19:57.419929063 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0327 19:19:57.430691 651901 docker.go:295] overlay module found
I0327 19:19:57.433221 651901 out.go:177] * Using the docker driver based on user configuration
I0327 19:19:57.435454 651901 start.go:297] selected driver: docker
I0327 19:19:57.435478 651901 start.go:901] validating driver "docker" against <nil>
I0327 19:19:57.435493 651901 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0327 19:19:57.436221 651901 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0327 19:19:57.505251 651901 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-03-27 19:19:57.496039194 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0327 19:19:57.505462 651901 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0327 19:19:57.505702 651901 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0327 19:19:57.507926 651901 out.go:177] * Using Docker driver with root privileges
I0327 19:19:57.509846 651901 cni.go:84] Creating CNI manager for ""
I0327 19:19:57.509874 651901 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0327 19:19:57.509884 651901 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0327 19:19:57.509967 651901 start.go:340] cluster config:
{Name:embed-certs-328500 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-328500 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:contain
erd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock:
SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0327 19:19:57.512055 651901 out.go:177] * Starting "embed-certs-328500" primary control-plane node in "embed-certs-328500" cluster
I0327 19:19:57.514091 651901 cache.go:121] Beginning downloading kic base image for docker with containerd
I0327 19:19:57.516495 651901 out.go:177] * Pulling base image v0.0.43-beta.0 ...
I0327 19:19:57.518544 651901 image.go:79] Checking for gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 in local docker daemon
I0327 19:19:57.518607 651901 preload.go:132] Checking if preload exists for k8s version v1.29.3 and runtime containerd
I0327 19:19:57.518656 651901 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-containerd-overlay2-arm64.tar.lz4
I0327 19:19:57.518666 651901 cache.go:56] Caching tarball of preloaded images
I0327 19:19:57.518747 651901 preload.go:173] Found /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0327 19:19:57.518754 651901 cache.go:59] Finished verifying existence of preloaded tar for v1.29.3 on containerd
I0327 19:19:57.518882 651901 profile.go:142] Saving config to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/config.json ...
I0327 19:19:57.518900 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/config.json: {Name:mk4e0e73dda9e42d352f7011deeae74a233a5ac6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:19:57.533098 651901 image.go:83] Found gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 in local docker daemon, skipping pull
I0327 19:19:57.533125 651901 cache.go:144] gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 exists in daemon, skipping load
I0327 19:19:57.533146 651901 cache.go:194] Successfully downloaded all kic artifacts
I0327 19:19:57.533176 651901 start.go:360] acquireMachinesLock for embed-certs-328500: {Name:mk1396a0195eb9c239c1f3f103b7f9cf04392dcd Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0327 19:19:57.533321 651901 start.go:364] duration metric: took 121.038µs to acquireMachinesLock for "embed-certs-328500"
I0327 19:19:57.533358 651901 start.go:93] Provisioning new machine with config: &{Name:embed-certs-328500 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-328500 Namespace:default APIServerHAVIP: APIServerName:minikubeCA
APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false Cust
omQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0327 19:19:57.533451 651901 start.go:125] createHost starting for "" (driver="docker")
I0327 19:19:56.091060 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:58.589083 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:00.617404 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:19:57.535817 651901 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0327 19:19:57.536069 651901 start.go:159] libmachine.API.Create for "embed-certs-328500" (driver="docker")
I0327 19:19:57.536104 651901 client.go:168] LocalClient.Create starting
I0327 19:19:57.536170 651901 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem
I0327 19:19:57.536210 651901 main.go:141] libmachine: Decoding PEM data...
I0327 19:19:57.536231 651901 main.go:141] libmachine: Parsing certificate...
I0327 19:19:57.536294 651901 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem
I0327 19:19:57.536318 651901 main.go:141] libmachine: Decoding PEM data...
I0327 19:19:57.536328 651901 main.go:141] libmachine: Parsing certificate...
I0327 19:19:57.536695 651901 cli_runner.go:164] Run: docker network inspect embed-certs-328500 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0327 19:19:57.550285 651901 cli_runner.go:211] docker network inspect embed-certs-328500 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0327 19:19:57.550393 651901 network_create.go:281] running [docker network inspect embed-certs-328500] to gather additional debugging logs...
I0327 19:19:57.550456 651901 cli_runner.go:164] Run: docker network inspect embed-certs-328500
W0327 19:19:57.563433 651901 cli_runner.go:211] docker network inspect embed-certs-328500 returned with exit code 1
I0327 19:19:57.563466 651901 network_create.go:284] error running [docker network inspect embed-certs-328500]: docker network inspect embed-certs-328500: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-328500 not found
I0327 19:19:57.563479 651901 network_create.go:286] output of [docker network inspect embed-certs-328500]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-328500 not found
** /stderr **
I0327 19:19:57.563590 651901 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0327 19:19:57.577250 651901 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-5bf317efd87b IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:a8:0f:db:92} reservation:<nil>}
I0327 19:19:57.577623 651901 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-f48867545a17 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:18:90:79:67} reservation:<nil>}
I0327 19:19:57.577979 651901 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-5c7f782bf7d2 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:ce:ac:8f:6f} reservation:<nil>}
I0327 19:19:57.578596 651901 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4002519d40}
I0327 19:19:57.578621 651901 network_create.go:124] attempt to create docker network embed-certs-328500 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0327 19:19:57.578692 651901 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-328500 embed-certs-328500
I0327 19:19:57.645061 651901 network_create.go:108] docker network embed-certs-328500 192.168.76.0/24 created
I0327 19:19:57.645091 651901 kic.go:121] calculated static IP "192.168.76.2" for the "embed-certs-328500" container
I0327 19:19:57.645163 651901 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0327 19:19:57.659301 651901 cli_runner.go:164] Run: docker volume create embed-certs-328500 --label name.minikube.sigs.k8s.io=embed-certs-328500 --label created_by.minikube.sigs.k8s.io=true
I0327 19:19:57.674540 651901 oci.go:103] Successfully created a docker volume embed-certs-328500
I0327 19:19:57.674645 651901 cli_runner.go:164] Run: docker run --rm --name embed-certs-328500-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-328500 --entrypoint /usr/bin/test -v embed-certs-328500:/var gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 -d /var/lib
I0327 19:19:58.218469 651901 oci.go:107] Successfully prepared a docker volume embed-certs-328500
I0327 19:19:58.218533 651901 preload.go:132] Checking if preload exists for k8s version v1.29.3 and runtime containerd
I0327 19:19:58.218554 651901 kic.go:194] Starting extracting preloaded images to volume ...
I0327 19:19:58.218632 651901 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-328500:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 -I lz4 -xf /preloaded.tar -C /extractDir
I0327 19:20:03.090739 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:05.119152 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:03.977539 651901 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18517-439728/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-328500:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 -I lz4 -xf /preloaded.tar -C /extractDir: (5.758868073s)
I0327 19:20:03.977575 651901 kic.go:203] duration metric: took 5.759017533s to extract preloaded images to volume ...
W0327 19:20:03.977722 651901 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0327 19:20:03.977837 651901 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0327 19:20:04.049481 651901 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-328500 --name embed-certs-328500 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-328500 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-328500 --network embed-certs-328500 --ip 192.168.76.2 --volume embed-certs-328500:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8
I0327 19:20:04.376395 651901 cli_runner.go:164] Run: docker container inspect embed-certs-328500 --format={{.State.Running}}
I0327 19:20:04.394337 651901 cli_runner.go:164] Run: docker container inspect embed-certs-328500 --format={{.State.Status}}
I0327 19:20:04.416071 651901 cli_runner.go:164] Run: docker exec embed-certs-328500 stat /var/lib/dpkg/alternatives/iptables
I0327 19:20:04.496080 651901 oci.go:144] the created container "embed-certs-328500" has a running status.
I0327 19:20:04.496106 651901 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa...
I0327 19:20:05.055393 651901 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0327 19:20:05.083796 651901 cli_runner.go:164] Run: docker container inspect embed-certs-328500 --format={{.State.Status}}
I0327 19:20:05.102981 651901 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0327 19:20:05.102999 651901 kic_runner.go:114] Args: [docker exec --privileged embed-certs-328500 chown docker:docker /home/docker/.ssh/authorized_keys]
I0327 19:20:05.189246 651901 cli_runner.go:164] Run: docker container inspect embed-certs-328500 --format={{.State.Status}}
I0327 19:20:05.225489 651901 machine.go:94] provisionDockerMachine start ...
I0327 19:20:05.225597 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:05.250072 651901 main.go:141] libmachine: Using SSH client type: native
I0327 19:20:05.250384 651901 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33472 <nil> <nil>}
I0327 19:20:05.250398 651901 main.go:141] libmachine: About to run SSH command:
hostname
I0327 19:20:05.418359 651901 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-328500
I0327 19:20:05.418381 651901 ubuntu.go:169] provisioning hostname "embed-certs-328500"
I0327 19:20:05.418488 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:05.434334 651901 main.go:141] libmachine: Using SSH client type: native
I0327 19:20:05.434642 651901 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33472 <nil> <nil>}
I0327 19:20:05.434658 651901 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-328500 && echo "embed-certs-328500" | sudo tee /etc/hostname
I0327 19:20:05.614196 651901 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-328500
I0327 19:20:05.614322 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:05.632240 651901 main.go:141] libmachine: Using SSH client type: native
I0327 19:20:05.632494 651901 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 33472 <nil> <nil>}
I0327 19:20:05.632520 651901 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-328500' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-328500/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-328500' | sudo tee -a /etc/hosts;
fi
fi
I0327 19:20:05.766730 651901 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0327 19:20:05.766757 651901 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18517-439728/.minikube CaCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18517-439728/.minikube}
I0327 19:20:05.766791 651901 ubuntu.go:177] setting up certificates
I0327 19:20:05.766812 651901 provision.go:84] configureAuth start
I0327 19:20:05.766872 651901 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-328500
I0327 19:20:05.789414 651901 provision.go:143] copyHostCerts
I0327 19:20:05.789599 651901 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem, removing ...
I0327 19:20:05.789617 651901 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem
I0327 19:20:05.789698 651901 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/ca.pem (1082 bytes)
I0327 19:20:05.789800 651901 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem, removing ...
I0327 19:20:05.789806 651901 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem
I0327 19:20:05.789834 651901 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/cert.pem (1123 bytes)
I0327 19:20:05.789891 651901 exec_runner.go:144] found /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem, removing ...
I0327 19:20:05.789896 651901 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem
I0327 19:20:05.789920 651901 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18517-439728/.minikube/key.pem (1675 bytes)
I0327 19:20:05.789972 651901 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem org=jenkins.embed-certs-328500 san=[127.0.0.1 192.168.76.2 embed-certs-328500 localhost minikube]
I0327 19:20:06.431599 651901 provision.go:177] copyRemoteCerts
I0327 19:20:06.431691 651901 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0327 19:20:06.431747 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:06.450967 651901 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33472 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa Username:docker}
I0327 19:20:06.544196 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0327 19:20:06.570985 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0327 19:20:06.599863 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0327 19:20:06.625400 651901 provision.go:87] duration metric: took 858.574688ms to configureAuth
I0327 19:20:06.625472 651901 ubuntu.go:193] setting minikube options for container-runtime
I0327 19:20:06.625678 651901 config.go:182] Loaded profile config "embed-certs-328500": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.29.3
I0327 19:20:06.625693 651901 machine.go:97] duration metric: took 1.400182594s to provisionDockerMachine
I0327 19:20:06.625702 651901 client.go:171] duration metric: took 9.08959059s to LocalClient.Create
I0327 19:20:06.625717 651901 start.go:167] duration metric: took 9.089648205s to libmachine.API.Create "embed-certs-328500"
I0327 19:20:06.625724 651901 start.go:293] postStartSetup for "embed-certs-328500" (driver="docker")
I0327 19:20:06.625739 651901 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0327 19:20:06.625807 651901 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0327 19:20:06.625849 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:06.643554 651901 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33472 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa Username:docker}
I0327 19:20:06.739803 651901 ssh_runner.go:195] Run: cat /etc/os-release
I0327 19:20:06.743058 651901 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0327 19:20:06.743095 651901 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0327 19:20:06.743106 651901 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0327 19:20:06.743113 651901 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0327 19:20:06.743139 651901 filesync.go:126] Scanning /home/jenkins/minikube-integration/18517-439728/.minikube/addons for local assets ...
I0327 19:20:06.743198 651901 filesync.go:126] Scanning /home/jenkins/minikube-integration/18517-439728/.minikube/files for local assets ...
I0327 19:20:06.743275 651901 filesync.go:149] local asset: /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem -> 4451462.pem in /etc/ssl/certs
I0327 19:20:06.743391 651901 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0327 19:20:06.752262 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem --> /etc/ssl/certs/4451462.pem (1708 bytes)
I0327 19:20:06.777762 651901 start.go:296] duration metric: took 152.002178ms for postStartSetup
I0327 19:20:06.778123 651901 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-328500
I0327 19:20:06.792828 651901 profile.go:142] Saving config to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/config.json ...
I0327 19:20:06.793125 651901 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0327 19:20:06.793191 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:06.807862 651901 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33472 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa Username:docker}
I0327 19:20:06.899309 651901 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0327 19:20:06.904072 651901 start.go:128] duration metric: took 9.370589242s to createHost
I0327 19:20:06.904159 651901 start.go:83] releasing machines lock for "embed-certs-328500", held for 9.370821728s
I0327 19:20:06.904275 651901 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-328500
I0327 19:20:06.919664 651901 ssh_runner.go:195] Run: cat /version.json
I0327 19:20:06.919694 651901 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0327 19:20:06.919715 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:06.919748 651901 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-328500
I0327 19:20:06.941143 651901 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33472 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa Username:docker}
I0327 19:20:06.941169 651901 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33472 SSHKeyPath:/home/jenkins/minikube-integration/18517-439728/.minikube/machines/embed-certs-328500/id_rsa Username:docker}
I0327 19:20:07.030065 651901 ssh_runner.go:195] Run: systemctl --version
I0327 19:20:07.159978 651901 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0327 19:20:07.164406 651901 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0327 19:20:07.189159 651901 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0327 19:20:07.189284 651901 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0327 19:20:07.221150 651901 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0327 19:20:07.221173 651901 start.go:494] detecting cgroup driver to use...
I0327 19:20:07.221219 651901 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0327 19:20:07.221275 651901 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0327 19:20:07.233586 651901 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0327 19:20:07.245104 651901 docker.go:217] disabling cri-docker service (if available) ...
I0327 19:20:07.245202 651901 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0327 19:20:07.259163 651901 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0327 19:20:07.273824 651901 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0327 19:20:07.380888 651901 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0327 19:20:07.500997 651901 docker.go:233] disabling docker service ...
I0327 19:20:07.501108 651901 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0327 19:20:07.524796 651901 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0327 19:20:07.538027 651901 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0327 19:20:07.642249 651901 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0327 19:20:07.729141 651901 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0327 19:20:07.740907 651901 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0327 19:20:07.758587 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I0327 19:20:07.769388 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0327 19:20:07.779824 651901 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0327 19:20:07.779935 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0327 19:20:07.790482 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0327 19:20:07.800458 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0327 19:20:07.811461 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0327 19:20:07.822517 651901 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0327 19:20:07.832111 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0327 19:20:07.842604 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0327 19:20:07.853400 651901 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0327 19:20:07.863622 651901 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0327 19:20:07.873269 651901 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0327 19:20:07.883181 651901 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0327 19:20:07.975181 651901 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0327 19:20:08.132376 651901 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0327 19:20:08.132506 651901 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0327 19:20:08.136407 651901 start.go:562] Will wait 60s for crictl version
I0327 19:20:08.136509 651901 ssh_runner.go:195] Run: which crictl
I0327 19:20:08.140073 651901 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0327 19:20:08.187276 651901 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0327 19:20:08.187405 651901 ssh_runner.go:195] Run: containerd --version
I0327 19:20:08.209291 651901 ssh_runner.go:195] Run: containerd --version
I0327 19:20:08.233294 651901 out.go:177] * Preparing Kubernetes v1.29.3 on containerd 1.6.28 ...
I0327 19:20:07.588744 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:09.590292 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:08.234850 651901 cli_runner.go:164] Run: docker network inspect embed-certs-328500 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0327 19:20:08.249662 651901 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0327 19:20:08.253442 651901 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0327 19:20:08.265054 651901 kubeadm.go:877] updating cluster {Name:embed-certs-328500 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-328500 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] A
PIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuF
irmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0327 19:20:08.265176 651901 preload.go:132] Checking if preload exists for k8s version v1.29.3 and runtime containerd
I0327 19:20:08.265236 651901 ssh_runner.go:195] Run: sudo crictl images --output json
I0327 19:20:08.302786 651901 containerd.go:627] all images are preloaded for containerd runtime.
I0327 19:20:08.302813 651901 containerd.go:534] Images already preloaded, skipping extraction
I0327 19:20:08.302878 651901 ssh_runner.go:195] Run: sudo crictl images --output json
I0327 19:20:08.341798 651901 containerd.go:627] all images are preloaded for containerd runtime.
I0327 19:20:08.341823 651901 cache_images.go:84] Images are preloaded, skipping loading
I0327 19:20:08.341832 651901 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.29.3 containerd true true} ...
I0327 19:20:08.341938 651901 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.29.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-328500 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.29.3 ClusterName:embed-certs-328500 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0327 19:20:08.342012 651901 ssh_runner.go:195] Run: sudo crictl info
I0327 19:20:08.380083 651901 cni.go:84] Creating CNI manager for ""
I0327 19:20:08.380104 651901 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0327 19:20:08.380117 651901 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0327 19:20:08.380140 651901 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.29.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-328500 NodeName:embed-certs-328500 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0327 19:20:08.380293 651901 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-328500"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.29.3
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0327 19:20:08.380369 651901 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.29.3
I0327 19:20:08.389904 651901 binaries.go:44] Found k8s binaries, skipping transfer
I0327 19:20:08.389975 651901 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0327 19:20:08.399302 651901 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I0327 19:20:08.418123 651901 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0327 19:20:08.438315 651901 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I0327 19:20:08.460185 651901 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0327 19:20:08.463783 651901 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0327 19:20:08.474963 651901 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0327 19:20:08.566627 651901 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0327 19:20:08.583124 651901 certs.go:68] Setting up /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500 for IP: 192.168.76.2
I0327 19:20:08.583149 651901 certs.go:194] generating shared ca certs ...
I0327 19:20:08.583165 651901 certs.go:226] acquiring lock for ca certs: {Name:mke441edb53252dfa332e7961e8e7663942b248d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:08.583317 651901 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18517-439728/.minikube/ca.key
I0327 19:20:08.583370 651901 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.key
I0327 19:20:08.583387 651901 certs.go:256] generating profile certs ...
I0327 19:20:08.583454 651901 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.key
I0327 19:20:08.583474 651901 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.crt with IP's: []
I0327 19:20:09.242381 651901 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.crt ...
I0327 19:20:09.242429 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.crt: {Name:mk66bbb8ac7c840072870c095cbf20826a1c06a7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:09.243270 651901 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.key ...
I0327 19:20:09.243289 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/client.key: {Name:mk2fd3c3f65181b38448152c8fad423ea5087686 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:09.243388 651901 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key.6adec09b
I0327 19:20:09.243408 651901 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt.6adec09b with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.76.2]
I0327 19:20:10.373505 651901 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt.6adec09b ...
I0327 19:20:10.373536 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt.6adec09b: {Name:mk6ab016d8e48b05a81c679701683812d84f478b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:10.374107 651901 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key.6adec09b ...
I0327 19:20:10.374126 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key.6adec09b: {Name:mk07712e9318ccf84ecf1088f278165994b730fd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:10.374215 651901 certs.go:381] copying /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt.6adec09b -> /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt
I0327 19:20:10.374303 651901 certs.go:385] copying /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key.6adec09b -> /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key
I0327 19:20:10.374367 651901 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.key
I0327 19:20:10.374384 651901 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.crt with IP's: []
I0327 19:20:10.555287 651901 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.crt ...
I0327 19:20:10.555317 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.crt: {Name:mk54a757581d2eef584142eebd8479d7094f42fd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:10.555900 651901 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.key ...
I0327 19:20:10.555916 651901 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.key: {Name:mk03967ac3bf64717002c76e8f396a3930435761 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0327 19:20:10.556541 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146.pem (1338 bytes)
W0327 19:20:10.556588 651901 certs.go:480] ignoring /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146_empty.pem, impossibly tiny 0 bytes
I0327 19:20:10.556601 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca-key.pem (1679 bytes)
I0327 19:20:10.556626 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/ca.pem (1082 bytes)
I0327 19:20:10.556651 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/cert.pem (1123 bytes)
I0327 19:20:10.556677 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/certs/key.pem (1675 bytes)
I0327 19:20:10.556727 651901 certs.go:484] found cert: /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem (1708 bytes)
I0327 19:20:10.557364 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0327 19:20:10.582262 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0327 19:20:10.609699 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0327 19:20:10.635106 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0327 19:20:10.660356 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0327 19:20:10.685220 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0327 19:20:10.709384 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0327 19:20:10.734039 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/profiles/embed-certs-328500/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0327 19:20:10.760285 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0327 19:20:10.785429 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/certs/445146.pem --> /usr/share/ca-certificates/445146.pem (1338 bytes)
I0327 19:20:10.810561 651901 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18517-439728/.minikube/files/etc/ssl/certs/4451462.pem --> /usr/share/ca-certificates/4451462.pem (1708 bytes)
I0327 19:20:10.835421 651901 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0327 19:20:10.853770 651901 ssh_runner.go:195] Run: openssl version
I0327 19:20:10.860651 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0327 19:20:10.870730 651901 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0327 19:20:10.874308 651901 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 27 18:28 /usr/share/ca-certificates/minikubeCA.pem
I0327 19:20:10.874443 651901 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0327 19:20:10.881497 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0327 19:20:10.891382 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/445146.pem && ln -fs /usr/share/ca-certificates/445146.pem /etc/ssl/certs/445146.pem"
I0327 19:20:10.900853 651901 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/445146.pem
I0327 19:20:10.904423 651901 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 27 18:34 /usr/share/ca-certificates/445146.pem
I0327 19:20:10.904495 651901 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/445146.pem
I0327 19:20:10.911534 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/445146.pem /etc/ssl/certs/51391683.0"
I0327 19:20:10.921060 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4451462.pem && ln -fs /usr/share/ca-certificates/4451462.pem /etc/ssl/certs/4451462.pem"
I0327 19:20:10.930566 651901 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4451462.pem
I0327 19:20:10.934005 651901 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 27 18:34 /usr/share/ca-certificates/4451462.pem
I0327 19:20:10.934078 651901 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4451462.pem
I0327 19:20:10.940916 651901 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4451462.pem /etc/ssl/certs/3ec20f2e.0"
I0327 19:20:10.950145 651901 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0327 19:20:10.955131 651901 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0327 19:20:10.955193 651901 kubeadm.go:391] StartCluster: {Name:embed-certs-328500 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.43-beta.0@sha256:185c97a62a2e62a78b853e29e445f05ffbcf36149614c192af3643aa3888c4e8 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-328500 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIS
erverIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirm
warePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0327 19:20:10.955279 651901 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0327 19:20:10.955341 651901 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0327 19:20:11.015493 651901 cri.go:89] found id: ""
I0327 19:20:11.015628 651901 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0327 19:20:11.027311 651901 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0327 19:20:11.036777 651901 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver
I0327 19:20:11.036846 651901 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0327 19:20:11.046667 651901 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0327 19:20:11.046691 651901 kubeadm.go:156] found existing configuration files:
I0327 19:20:11.046798 651901 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0327 19:20:11.056240 651901 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0327 19:20:11.056371 651901 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0327 19:20:11.065509 651901 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0327 19:20:11.075331 651901 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0327 19:20:11.075448 651901 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0327 19:20:11.087468 651901 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0327 19:20:11.099032 651901 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0327 19:20:11.099107 651901 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0327 19:20:11.108729 651901 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0327 19:20:11.119111 651901 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0327 19:20:11.119212 651901 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0327 19:20:11.128324 651901 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.29.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0327 19:20:11.177941 651901 kubeadm.go:309] [init] Using Kubernetes version: v1.29.3
I0327 19:20:11.178145 651901 kubeadm.go:309] [preflight] Running pre-flight checks
I0327 19:20:11.221119 651901 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:
I0327 19:20:11.221195 651901 kubeadm.go:309] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1056-aws[0m
I0327 19:20:11.221234 651901 kubeadm.go:309] [0;37mOS[0m: [0;32mLinux[0m
I0327 19:20:11.221282 651901 kubeadm.go:309] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0327 19:20:11.221354 651901 kubeadm.go:309] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0327 19:20:11.221407 651901 kubeadm.go:309] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0327 19:20:11.221457 651901 kubeadm.go:309] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0327 19:20:11.221507 651901 kubeadm.go:309] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0327 19:20:11.221563 651901 kubeadm.go:309] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0327 19:20:11.221611 651901 kubeadm.go:309] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0327 19:20:11.221660 651901 kubeadm.go:309] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0327 19:20:11.221708 651901 kubeadm.go:309] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0327 19:20:11.294035 651901 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0327 19:20:11.294145 651901 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0327 19:20:11.294242 651901 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0327 19:20:11.548538 651901 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0327 19:20:11.552543 651901 out.go:204] - Generating certificates and keys ...
I0327 19:20:11.552683 651901 kubeadm.go:309] [certs] Using existing ca certificate authority
I0327 19:20:11.552776 651901 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0327 19:20:12.091204 642122 pod_ready.go:102] pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace has status "Ready":"False"
I0327 19:20:12.591512 642122 pod_ready.go:81] duration metric: took 4m0.009346666s for pod "metrics-server-9975d5f86-p8k5b" in "kube-system" namespace to be "Ready" ...
E0327 19:20:12.591546 642122 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0327 19:20:12.591555 642122 pod_ready.go:38] duration metric: took 5m20.881706224s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0327 19:20:12.591573 642122 api_server.go:52] waiting for apiserver process to appear ...
I0327 19:20:12.591611 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0327 19:20:12.591681 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0327 19:20:12.661305 642122 cri.go:89] found id: "0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:12.661383 642122 cri.go:89] found id: "2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:12.661391 642122 cri.go:89] found id: ""
I0327 19:20:12.661399 642122 logs.go:276] 2 containers: [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c]
I0327 19:20:12.661490 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.665803 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.669847 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0327 19:20:12.669969 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0327 19:20:12.722052 642122 cri.go:89] found id: "52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:12.722121 642122 cri.go:89] found id: "2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:12.722139 642122 cri.go:89] found id: ""
I0327 19:20:12.722162 642122 logs.go:276] 2 containers: [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42]
I0327 19:20:12.722248 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.726345 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.730599 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0327 19:20:12.730733 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0327 19:20:12.805110 642122 cri.go:89] found id: "6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:12.805182 642122 cri.go:89] found id: "1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:12.805200 642122 cri.go:89] found id: ""
I0327 19:20:12.805225 642122 logs.go:276] 2 containers: [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b]
I0327 19:20:12.805323 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.809380 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.814289 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0327 19:20:12.814427 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0327 19:20:12.864113 642122 cri.go:89] found id: "45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:12.864193 642122 cri.go:89] found id: "35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:12.864212 642122 cri.go:89] found id: ""
I0327 19:20:12.864235 642122 logs.go:276] 2 containers: [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424]
I0327 19:20:12.864322 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.868917 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.873038 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0327 19:20:12.873178 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0327 19:20:12.923144 642122 cri.go:89] found id: "de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:12.923200 642122 cri.go:89] found id: "2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:12.923218 642122 cri.go:89] found id: ""
I0327 19:20:12.923240 642122 logs.go:276] 2 containers: [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83]
I0327 19:20:12.923324 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.927187 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:12.931355 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0327 19:20:12.931473 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0327 19:20:13.003546 642122 cri.go:89] found id: "6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:13.003627 642122 cri.go:89] found id: "1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:13.003646 642122 cri.go:89] found id: ""
I0327 19:20:13.003671 642122 logs.go:276] 2 containers: [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b]
I0327 19:20:13.003771 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.008616 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.018497 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0327 19:20:13.018630 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0327 19:20:13.085416 642122 cri.go:89] found id: "207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:13.085485 642122 cri.go:89] found id: "63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:13.085502 642122 cri.go:89] found id: ""
I0327 19:20:13.085525 642122 logs.go:276] 2 containers: [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737]
I0327 19:20:13.085611 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.090248 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.094274 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0327 19:20:13.094442 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0327 19:20:13.152485 642122 cri.go:89] found id: "63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:13.152547 642122 cri.go:89] found id: "748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:13.152578 642122 cri.go:89] found id: ""
I0327 19:20:13.152602 642122 logs.go:276] 2 containers: [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448]
I0327 19:20:13.152688 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.156834 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.160902 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0327 19:20:13.161040 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0327 19:20:13.213344 642122 cri.go:89] found id: "44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:13.213406 642122 cri.go:89] found id: ""
I0327 19:20:13.213439 642122 logs.go:276] 1 containers: [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585]
I0327 19:20:13.213526 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:13.221951 642122 logs.go:123] Gathering logs for etcd [2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42] ...
I0327 19:20:13.222022 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:13.297783 642122 logs.go:123] Gathering logs for storage-provisioner [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07] ...
I0327 19:20:13.297859 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:13.348878 642122 logs.go:123] Gathering logs for storage-provisioner [748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448] ...
I0327 19:20:13.348955 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:13.404969 642122 logs.go:123] Gathering logs for containerd ...
I0327 19:20:13.405044 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0327 19:20:13.465407 642122 logs.go:123] Gathering logs for container status ...
I0327 19:20:13.465484 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0327 19:20:13.541221 642122 logs.go:123] Gathering logs for coredns [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228] ...
I0327 19:20:13.541381 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:13.587393 642122 logs.go:123] Gathering logs for kube-scheduler [35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424] ...
I0327 19:20:13.587459 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:13.645192 642122 logs.go:123] Gathering logs for kubernetes-dashboard [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585] ...
I0327 19:20:13.645362 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:13.701791 642122 logs.go:123] Gathering logs for dmesg ...
I0327 19:20:13.701857 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0327 19:20:13.722862 642122 logs.go:123] Gathering logs for kube-apiserver [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68] ...
I0327 19:20:13.722982 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:13.814163 642122 logs.go:123] Gathering logs for kube-apiserver [2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c] ...
I0327 19:20:13.814197 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:13.876593 642122 logs.go:123] Gathering logs for etcd [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb] ...
I0327 19:20:13.876627 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:13.984118 642122 logs.go:123] Gathering logs for kube-scheduler [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e] ...
I0327 19:20:13.984151 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:14.056761 642122 logs.go:123] Gathering logs for kube-proxy [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5] ...
I0327 19:20:14.056833 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:14.112348 642122 logs.go:123] Gathering logs for kindnet [63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737] ...
I0327 19:20:14.112379 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:14.164125 642122 logs.go:123] Gathering logs for kubelet ...
I0327 19:20:14.164159 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0327 19:20:14.226581 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.640363 662 reflector.go:138] object-"kube-system"/"metrics-server-token-bntmf": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-bntmf" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227269 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686154 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8c799": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8c799" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227614 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686269 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-qwbnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-qwbnq" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.227851 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686334 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228080 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686386 662 reflector.go:138] object-"default"/"default-token-l225m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-l225m" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228300 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688291 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228526 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688368 662 reflector.go:138] object-"kube-system"/"coredns-token-h758g": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-h758g" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.228754 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688486 662 reflector.go:138] object-"kube-system"/"kindnet-token-pvkpn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pvkpn" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:14.236452 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.420152 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.236691 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.612018 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.241434 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:04 old-k8s-version-152774 kubelet[662]: E0327 19:15:04.457953 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.243155 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:15 old-k8s-version-152774 kubelet[662]: E0327 19:15:15.468690 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.243759 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:21 old-k8s-version-152774 kubelet[662]: E0327 19:15:21.722972 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.244105 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:22 old-k8s-version-152774 kubelet[662]: E0327 19:15:22.735339 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.244815 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:25 old-k8s-version-152774 kubelet[662]: E0327 19:15:25.475266 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.247381 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:27 old-k8s-version-152774 kubelet[662]: E0327 19:15:27.470008 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.248331 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:37 old-k8s-version-152774 kubelet[662]: E0327 19:15:37.769827 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.248554 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:40 old-k8s-version-152774 kubelet[662]: E0327 19:15:40.468375 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.248901 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:45 old-k8s-version-152774 kubelet[662]: E0327 19:15:45.475591 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.249104 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:55 old-k8s-version-152774 kubelet[662]: E0327 19:15:55.447878 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.249452 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:56 old-k8s-version-152774 kubelet[662]: E0327 19:15:56.447736 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.252060 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.478662 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.252409 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.843057 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.252903 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:15 old-k8s-version-152774 kubelet[662]: E0327 19:16:15.475201 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.253116 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:23 old-k8s-version-152774 kubelet[662]: E0327 19:16:23.455765 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.253468 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:26 old-k8s-version-152774 kubelet[662]: E0327 19:16:26.447884 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.253672 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:34 old-k8s-version-152774 kubelet[662]: E0327 19:16:34.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.254015 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:41 old-k8s-version-152774 kubelet[662]: E0327 19:16:41.447929 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.254220 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:45 old-k8s-version-152774 kubelet[662]: E0327 19:16:45.448175 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.254830 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:55 old-k8s-version-152774 kubelet[662]: E0327 19:16:55.932646 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.255039 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:59 old-k8s-version-152774 kubelet[662]: E0327 19:16:59.447909 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.255382 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:05 old-k8s-version-152774 kubelet[662]: E0327 19:17:05.475157 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.255584 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:14 old-k8s-version-152774 kubelet[662]: E0327 19:17:14.447954 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.256002 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:17 old-k8s-version-152774 kubelet[662]: E0327 19:17:17.447555 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.256209 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:25 old-k8s-version-152774 kubelet[662]: E0327 19:17:25.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.256552 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:28 old-k8s-version-152774 kubelet[662]: E0327 19:17:28.447656 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.259030 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:40 old-k8s-version-152774 kubelet[662]: E0327 19:17:40.456218 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:14.259390 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:41 old-k8s-version-152774 kubelet[662]: E0327 19:17:41.447800 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.259602 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:53 old-k8s-version-152774 kubelet[662]: E0327 19:17:53.447859 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.259960 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:56 old-k8s-version-152774 kubelet[662]: E0327 19:17:56.447992 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.260161 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:04 old-k8s-version-152774 kubelet[662]: E0327 19:18:04.448632 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.260509 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:09 old-k8s-version-152774 kubelet[662]: E0327 19:18:09.447980 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.260710 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:16 old-k8s-version-152774 kubelet[662]: E0327 19:18:16.448113 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.261319 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:25 old-k8s-version-152774 kubelet[662]: E0327 19:18:25.174857 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.261668 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:26 old-k8s-version-152774 kubelet[662]: E0327 19:18:26.177914 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.261869 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:29 old-k8s-version-152774 kubelet[662]: E0327 19:18:29.447914 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.262211 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:41 old-k8s-version-152774 kubelet[662]: E0327 19:18:41.447526 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.262524 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:42 old-k8s-version-152774 kubelet[662]: E0327 19:18:42.449330 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.262734 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:54 old-k8s-version-152774 kubelet[662]: E0327 19:18:54.448024 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.263080 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:55 old-k8s-version-152774 kubelet[662]: E0327 19:18:55.447433 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.263284 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:08 old-k8s-version-152774 kubelet[662]: E0327 19:19:08.447831 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.263712 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: E0327 19:19:09.447537 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.263915 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:20 old-k8s-version-152774 kubelet[662]: E0327 19:19:20.448039 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.264260 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: E0327 19:19:21.447551 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.264602 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: E0327 19:19:32.448689 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.264805 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265147 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.265365 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265579 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.265930 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:14.265946 642122 logs.go:123] Gathering logs for describe nodes ...
I0327 19:20:14.265976 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0327 19:20:14.514548 642122 logs.go:123] Gathering logs for coredns [1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b] ...
I0327 19:20:14.514596 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:14.573317 642122 logs.go:123] Gathering logs for kube-proxy [2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83] ...
I0327 19:20:14.573352 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:14.624315 642122 logs.go:123] Gathering logs for kube-controller-manager [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f] ...
I0327 19:20:14.624344 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:14.717094 642122 logs.go:123] Gathering logs for kube-controller-manager [1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b] ...
I0327 19:20:14.717132 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:14.817864 642122 logs.go:123] Gathering logs for kindnet [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910] ...
I0327 19:20:14.817903 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:14.875620 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:14.875648 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0327 19:20:14.875726 642122 out.go:239] X Problems detected in kubelet:
W0327 19:20:14.875757 642122 out.go:239] Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875791 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:14.875807 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875846 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:14.875864 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:14.875877 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:14.875884 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:20:12.629246 651901 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0327 19:20:12.921610 651901 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0327 19:20:13.765755 651901 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0327 19:20:14.220057 651901 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0327 19:20:14.870344 651901 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0327 19:20:14.870920 651901 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [embed-certs-328500 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0327 19:20:15.199672 651901 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0327 19:20:15.200028 651901 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-328500 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0327 19:20:15.688439 651901 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0327 19:20:15.948806 651901 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0327 19:20:16.149703 651901 kubeadm.go:309] [certs] Generating "sa" key and public key
I0327 19:20:16.149951 651901 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0327 19:20:17.104377 651901 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0327 19:20:18.067674 651901 kubeadm.go:309] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I0327 19:20:18.212767 651901 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0327 19:20:19.268150 651901 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0327 19:20:19.658373 651901 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0327 19:20:19.659154 651901 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0327 19:20:19.663776 651901 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0327 19:20:19.666255 651901 out.go:204] - Booting up control plane ...
I0327 19:20:19.666352 651901 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0327 19:20:19.666454 651901 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0327 19:20:19.667173 651901 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0327 19:20:19.678464 651901 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0327 19:20:19.679376 651901 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0327 19:20:19.679620 651901 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0327 19:20:19.777225 651901 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0327 19:20:24.877342 642122 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0327 19:20:24.891922 642122 api_server.go:72] duration metric: took 5m50.263854092s to wait for apiserver process to appear ...
I0327 19:20:24.891946 642122 api_server.go:88] waiting for apiserver healthz status ...
I0327 19:20:24.891980 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0327 19:20:24.892035 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0327 19:20:24.974524 642122 cri.go:89] found id: "0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:24.974547 642122 cri.go:89] found id: "2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:24.974552 642122 cri.go:89] found id: ""
I0327 19:20:24.974560 642122 logs.go:276] 2 containers: [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c]
I0327 19:20:24.974621 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:24.980100 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:24.984691 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0327 19:20:24.984765 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0327 19:20:25.098328 642122 cri.go:89] found id: "52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:25.098353 642122 cri.go:89] found id: "2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:25.098358 642122 cri.go:89] found id: ""
I0327 19:20:25.098366 642122 logs.go:276] 2 containers: [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42]
I0327 19:20:25.098445 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.103699 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.110237 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0327 19:20:25.110313 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0327 19:20:25.194586 642122 cri.go:89] found id: "6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:25.194623 642122 cri.go:89] found id: "1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:25.194628 642122 cri.go:89] found id: ""
I0327 19:20:25.194636 642122 logs.go:276] 2 containers: [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b]
I0327 19:20:25.194704 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.199613 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.203738 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0327 19:20:25.203822 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0327 19:20:25.268088 642122 cri.go:89] found id: "45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:25.268115 642122 cri.go:89] found id: "35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:25.268120 642122 cri.go:89] found id: ""
I0327 19:20:25.268128 642122 logs.go:276] 2 containers: [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424]
I0327 19:20:25.268186 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.274153 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.278971 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0327 19:20:25.279050 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0327 19:20:25.386946 642122 cri.go:89] found id: "de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:25.386971 642122 cri.go:89] found id: "2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:25.386977 642122 cri.go:89] found id: ""
I0327 19:20:25.386984 642122 logs.go:276] 2 containers: [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83]
I0327 19:20:25.387041 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.393856 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.403946 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0327 19:20:25.404036 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0327 19:20:25.455380 642122 cri.go:89] found id: "6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:25.455403 642122 cri.go:89] found id: "1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:25.455408 642122 cri.go:89] found id: ""
I0327 19:20:25.455416 642122 logs.go:276] 2 containers: [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b]
I0327 19:20:25.455487 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.460988 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.465435 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0327 19:20:25.465532 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0327 19:20:25.545617 642122 cri.go:89] found id: "207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:25.545642 642122 cri.go:89] found id: "63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:25.545648 642122 cri.go:89] found id: ""
I0327 19:20:25.545665 642122 logs.go:276] 2 containers: [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737]
I0327 19:20:25.545722 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.549385 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.556895 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0327 19:20:25.556967 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0327 19:20:25.650130 642122 cri.go:89] found id: "63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:25.650154 642122 cri.go:89] found id: "748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:25.650160 642122 cri.go:89] found id: ""
I0327 19:20:25.650168 642122 logs.go:276] 2 containers: [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448]
I0327 19:20:25.650225 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.654924 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.658948 642122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0327 19:20:25.659030 642122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0327 19:20:25.718309 642122 cri.go:89] found id: "44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:25.718331 642122 cri.go:89] found id: ""
I0327 19:20:25.718340 642122 logs.go:276] 1 containers: [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585]
I0327 19:20:25.718454 642122 ssh_runner.go:195] Run: which crictl
I0327 19:20:25.724763 642122 logs.go:123] Gathering logs for storage-provisioner [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07] ...
I0327 19:20:25.724792 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07"
I0327 19:20:25.771906 642122 logs.go:123] Gathering logs for container status ...
I0327 19:20:25.771943 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0327 19:20:25.826538 642122 logs.go:123] Gathering logs for kubelet ...
I0327 19:20:25.826571 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0327 19:20:28.779717 651901 kubeadm.go:309] [apiclient] All control plane components are healthy after 9.002830 seconds
I0327 19:20:28.803116 651901 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0327 19:20:28.825576 651901 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0327 19:20:29.369016 651901 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs
I0327 19:20:29.369230 651901 kubeadm.go:309] [mark-control-plane] Marking the node embed-certs-328500 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0327 19:20:29.881712 651901 kubeadm.go:309] [bootstrap-token] Using token: jvfayf.k41egzbpno7fmj84
I0327 19:20:29.884136 651901 out.go:204] - Configuring RBAC rules ...
I0327 19:20:29.884268 651901 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0327 19:20:29.894684 651901 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0327 19:20:29.903581 651901 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0327 19:20:29.907839 651901 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0327 19:20:29.912972 651901 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0327 19:20:29.917211 651901 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0327 19:20:29.932221 651901 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0327 19:20:30.222209 651901 kubeadm.go:309] [addons] Applied essential addon: CoreDNS
I0327 19:20:30.307370 651901 kubeadm.go:309] [addons] Applied essential addon: kube-proxy
I0327 19:20:30.309088 651901 kubeadm.go:309]
I0327 19:20:30.309174 651901 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!
I0327 19:20:30.309184 651901 kubeadm.go:309]
I0327 19:20:30.309259 651901 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:
I0327 19:20:30.309276 651901 kubeadm.go:309]
I0327 19:20:30.309301 651901 kubeadm.go:309] mkdir -p $HOME/.kube
I0327 19:20:30.309715 651901 kubeadm.go:309] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0327 19:20:30.309789 651901 kubeadm.go:309] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0327 19:20:30.309800 651901 kubeadm.go:309]
I0327 19:20:30.309852 651901 kubeadm.go:309] Alternatively, if you are the root user, you can run:
I0327 19:20:30.309861 651901 kubeadm.go:309]
I0327 19:20:30.309906 651901 kubeadm.go:309] export KUBECONFIG=/etc/kubernetes/admin.conf
I0327 19:20:30.309914 651901 kubeadm.go:309]
I0327 19:20:30.309964 651901 kubeadm.go:309] You should now deploy a pod network to the cluster.
I0327 19:20:30.310045 651901 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0327 19:20:30.310117 651901 kubeadm.go:309] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0327 19:20:30.310126 651901 kubeadm.go:309]
I0327 19:20:30.310436 651901 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities
I0327 19:20:30.310523 651901 kubeadm.go:309] and service account keys on each node and then running the following as root:
I0327 19:20:30.310532 651901 kubeadm.go:309]
I0327 19:20:30.310802 651901 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token jvfayf.k41egzbpno7fmj84 \
I0327 19:20:30.310916 651901 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:b5bcb4a97240f3b480f86ebd66301c6ee86688c4a3458e4511adbbcedaf952ec \
I0327 19:20:30.311167 651901 kubeadm.go:309] --control-plane
I0327 19:20:30.311184 651901 kubeadm.go:309]
I0327 19:20:30.311446 651901 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:
I0327 19:20:30.311462 651901 kubeadm.go:309]
I0327 19:20:30.312214 651901 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token jvfayf.k41egzbpno7fmj84 \
I0327 19:20:30.312577 651901 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:b5bcb4a97240f3b480f86ebd66301c6ee86688c4a3458e4511adbbcedaf952ec
I0327 19:20:30.322366 651901 kubeadm.go:309] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1056-aws\n", err: exit status 1
I0327 19:20:30.322560 651901 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0327 19:20:30.322587 651901 cni.go:84] Creating CNI manager for ""
I0327 19:20:30.322600 651901 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0327 19:20:30.326958 651901 out.go:177] * Configuring CNI (Container Networking Interface) ...
W0327 19:20:25.895233 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.640363 662 reflector.go:138] object-"kube-system"/"metrics-server-token-bntmf": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-bntmf" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.895479 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686154 662 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8c799": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8c799" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.895697 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686269 662 reflector.go:138] object-"kube-system"/"kube-proxy-token-qwbnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-qwbnq" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.900828 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686334 662 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901067 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.686386 662 reflector.go:138] object-"default"/"default-token-l225m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-l225m" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901286 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688291 662 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901495 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688368 662 reflector.go:138] object-"kube-system"/"coredns-token-h758g": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-h758g" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.901706 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:51 old-k8s-version-152774 kubelet[662]: E0327 19:14:51.688486 662 reflector.go:138] object-"kube-system"/"kindnet-token-pvkpn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pvkpn" is forbidden: User "system:node:old-k8s-version-152774" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-152774' and this object
W0327 19:20:25.909567 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.420152 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.909765 642122 logs.go:138] Found kubelet problem: Mar 27 19:14:53 old-k8s-version-152774 kubelet[662]: E0327 19:14:53.612018 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.918075 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:04 old-k8s-version-152774 kubelet[662]: E0327 19:15:04.457953 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.919807 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:15 old-k8s-version-152774 kubelet[662]: E0327 19:15:15.468690 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.920412 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:21 old-k8s-version-152774 kubelet[662]: E0327 19:15:21.722972 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.920750 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:22 old-k8s-version-152774 kubelet[662]: E0327 19:15:22.735339 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.921410 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:25 old-k8s-version-152774 kubelet[662]: E0327 19:15:25.475266 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.923922 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:27 old-k8s-version-152774 kubelet[662]: E0327 19:15:27.470008 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.924881 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:37 old-k8s-version-152774 kubelet[662]: E0327 19:15:37.769827 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.925068 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:40 old-k8s-version-152774 kubelet[662]: E0327 19:15:40.468375 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.925395 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:45 old-k8s-version-152774 kubelet[662]: E0327 19:15:45.475591 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.925577 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:55 old-k8s-version-152774 kubelet[662]: E0327 19:15:55.447878 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.925900 642122 logs.go:138] Found kubelet problem: Mar 27 19:15:56 old-k8s-version-152774 kubelet[662]: E0327 19:15:56.447736 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.928453 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.478662 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.928780 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:10 old-k8s-version-152774 kubelet[662]: E0327 19:16:10.843057 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929232 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:15 old-k8s-version-152774 kubelet[662]: E0327 19:16:15.475201 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929421 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:23 old-k8s-version-152774 kubelet[662]: E0327 19:16:23.455765 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.929776 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:26 old-k8s-version-152774 kubelet[662]: E0327 19:16:26.447884 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.929965 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:34 old-k8s-version-152774 kubelet[662]: E0327 19:16:34.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.930292 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:41 old-k8s-version-152774 kubelet[662]: E0327 19:16:41.447929 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.930479 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:45 old-k8s-version-152774 kubelet[662]: E0327 19:16:45.448175 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.931141 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:55 old-k8s-version-152774 kubelet[662]: E0327 19:16:55.932646 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.931328 642122 logs.go:138] Found kubelet problem: Mar 27 19:16:59 old-k8s-version-152774 kubelet[662]: E0327 19:16:59.447909 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.931653 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:05 old-k8s-version-152774 kubelet[662]: E0327 19:17:05.475157 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.931835 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:14 old-k8s-version-152774 kubelet[662]: E0327 19:17:14.447954 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.932243 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:17 old-k8s-version-152774 kubelet[662]: E0327 19:17:17.447555 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.932432 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:25 old-k8s-version-152774 kubelet[662]: E0327 19:17:25.448040 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.932763 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:28 old-k8s-version-152774 kubelet[662]: E0327 19:17:28.447656 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.935363 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:40 old-k8s-version-152774 kubelet[662]: E0327 19:17:40.456218 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0327 19:20:25.935700 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:41 old-k8s-version-152774 kubelet[662]: E0327 19:17:41.447800 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.935885 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:53 old-k8s-version-152774 kubelet[662]: E0327 19:17:53.447859 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.936209 642122 logs.go:138] Found kubelet problem: Mar 27 19:17:56 old-k8s-version-152774 kubelet[662]: E0327 19:17:56.447992 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.936397 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:04 old-k8s-version-152774 kubelet[662]: E0327 19:18:04.448632 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.936720 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:09 old-k8s-version-152774 kubelet[662]: E0327 19:18:09.447980 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.936903 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:16 old-k8s-version-152774 kubelet[662]: E0327 19:18:16.448113 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.937518 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:25 old-k8s-version-152774 kubelet[662]: E0327 19:18:25.174857 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.937849 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:26 old-k8s-version-152774 kubelet[662]: E0327 19:18:26.177914 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.938100 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:29 old-k8s-version-152774 kubelet[662]: E0327 19:18:29.447914 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.938448 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:41 old-k8s-version-152774 kubelet[662]: E0327 19:18:41.447526 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.938648 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:42 old-k8s-version-152774 kubelet[662]: E0327 19:18:42.449330 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.938833 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:54 old-k8s-version-152774 kubelet[662]: E0327 19:18:54.448024 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.939160 642122 logs.go:138] Found kubelet problem: Mar 27 19:18:55 old-k8s-version-152774 kubelet[662]: E0327 19:18:55.447433 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.939343 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:08 old-k8s-version-152774 kubelet[662]: E0327 19:19:08.447831 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.939667 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: E0327 19:19:09.447537 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.939848 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:20 old-k8s-version-152774 kubelet[662]: E0327 19:19:20.448039 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.940171 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: E0327 19:19:21.447551 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.940503 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: E0327 19:19:32.448689 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.940685 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941011 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.941193 642122 logs.go:138] Found kubelet problem: Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941379 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.941701 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:25.942013 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:25.942207 642122 logs.go:138] Found kubelet problem: Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:25.942222 642122 logs.go:123] Gathering logs for dmesg ...
I0327 19:20:25.942236 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0327 19:20:25.979844 642122 logs.go:123] Gathering logs for etcd [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb] ...
I0327 19:20:25.979986 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb"
I0327 19:20:26.072884 642122 logs.go:123] Gathering logs for kube-controller-manager [1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b] ...
I0327 19:20:26.072923 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b"
I0327 19:20:26.172742 642122 logs.go:123] Gathering logs for kube-apiserver [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68] ...
I0327 19:20:26.172822 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68"
I0327 19:20:26.247706 642122 logs.go:123] Gathering logs for kube-apiserver [2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c] ...
I0327 19:20:26.247782 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c"
I0327 19:20:26.399253 642122 logs.go:123] Gathering logs for kube-proxy [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5] ...
I0327 19:20:26.399333 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5"
I0327 19:20:26.539830 642122 logs.go:123] Gathering logs for kindnet [63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737] ...
I0327 19:20:26.539856 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737"
I0327 19:20:26.631701 642122 logs.go:123] Gathering logs for kube-scheduler [35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424] ...
I0327 19:20:26.631776 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424"
I0327 19:20:26.731781 642122 logs.go:123] Gathering logs for kube-controller-manager [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f] ...
I0327 19:20:26.731977 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f"
I0327 19:20:26.859144 642122 logs.go:123] Gathering logs for describe nodes ...
I0327 19:20:26.859224 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0327 19:20:27.130798 642122 logs.go:123] Gathering logs for etcd [2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42] ...
I0327 19:20:27.130873 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42"
I0327 19:20:27.193000 642122 logs.go:123] Gathering logs for coredns [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228] ...
I0327 19:20:27.193110 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228"
I0327 19:20:27.246520 642122 logs.go:123] Gathering logs for coredns [1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b] ...
I0327 19:20:27.246600 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b"
I0327 19:20:27.295770 642122 logs.go:123] Gathering logs for kubernetes-dashboard [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585] ...
I0327 19:20:27.295842 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585"
I0327 19:20:27.365331 642122 logs.go:123] Gathering logs for containerd ...
I0327 19:20:27.365410 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0327 19:20:27.424989 642122 logs.go:123] Gathering logs for kube-scheduler [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e] ...
I0327 19:20:27.425063 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e"
I0327 19:20:27.475636 642122 logs.go:123] Gathering logs for kube-proxy [2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83] ...
I0327 19:20:27.475662 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83"
I0327 19:20:27.532399 642122 logs.go:123] Gathering logs for kindnet [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910] ...
I0327 19:20:27.532469 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910"
I0327 19:20:27.584156 642122 logs.go:123] Gathering logs for storage-provisioner [748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448] ...
I0327 19:20:27.584249 642122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448"
I0327 19:20:27.642669 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:27.642695 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0327 19:20:27.642759 642122 out.go:239] X Problems detected in kubelet:
W0327 19:20:27.642775 642122 out.go:239] Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642784 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642795 642122 out.go:239] Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
W0327 19:20:27.642818 642122 out.go:239] Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0327 19:20:27.642846 642122 out.go:239] Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
I0327 19:20:27.642861 642122 out.go:304] Setting ErrFile to fd 2...
I0327 19:20:27.642868 642122 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0327 19:20:30.329207 651901 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap
I0327 19:20:30.334299 651901 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.29.3/kubectl ...
I0327 19:20:30.334318 651901 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes)
I0327 19:20:30.374510 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0327 19:20:30.795270 651901 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0327 19:20:30.795414 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:30.795490 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes embed-certs-328500 minikube.k8s.io/updated_at=2024_03_27T19_20_30_0700 minikube.k8s.io/version=v1.33.0-beta.0 minikube.k8s.io/commit=475b39f6a1dc94a0c7060d2eec10d9b995edcd28 minikube.k8s.io/name=embed-certs-328500 minikube.k8s.io/primary=true
I0327 19:20:30.940078 651901 ops.go:34] apiserver oom_adj: -16
I0327 19:20:30.940171 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:31.440997 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:31.940795 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:32.441179 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:32.940309 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:33.441305 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:33.940868 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:34.441237 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:34.940565 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:35.440853 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:35.940513 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:36.440940 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:36.941244 651901 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.29.3/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0327 19:20:37.643979 642122 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0327 19:20:37.653515 642122 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0327 19:20:37.656458 642122 out.go:177]
W0327 19:20:37.658607 642122 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0327 19:20:37.658642 642122 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0327 19:20:37.658659 642122 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0327 19:20:37.658665 642122 out.go:239] *
W0327 19:20:37.659646 642122 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0327 19:20:37.662693 642122 out.go:177]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
a4f7bac13f4a7 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 1dd65b48d5af6 dashboard-metrics-scraper-8d5bb5db8-9ddc4
44d7fd2c3587b 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 7420c4c05094d kubernetes-dashboard-cd95d586-cchqs
63d5d568691f6 ba04bb24b9575 5 minutes ago Running storage-provisioner 1 8b917a83c3d95 storage-provisioner
207572546e278 4740c1948d3fc 5 minutes ago Running kindnet-cni 1 7cf41eefe31c6 kindnet-697g5
6befe5bb11708 db91994f4ee8f 5 minutes ago Running coredns 1 3815d0ea91895 coredns-74ff55c5b-p8r76
de25424b8f929 25a5233254979 5 minutes ago Running kube-proxy 1 70973a676d9c8 kube-proxy-9tnq4
81ef7cb26aed5 1611cd07b61d5 5 minutes ago Running busybox 1 e07772b6d222e busybox
0ffe2baaed512 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 6cf7d66184085 kube-apiserver-old-k8s-version-152774
45f861afe1590 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 afcd9f0fe0951 kube-scheduler-old-k8s-version-152774
52e0ba1f0d529 05b738aa1bc63 5 minutes ago Running etcd 1 9b4d311cd35d9 etcd-old-k8s-version-152774
6c75c3cfda9f5 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 e5b6ed4c0b86f kube-controller-manager-old-k8s-version-152774
68707e38ffc5d 1611cd07b61d5 6 minutes ago Exited busybox 0 9f2bb8008a118 busybox
1b4d47ebc9130 db91994f4ee8f 7 minutes ago Exited coredns 0 5dea29edb775a coredns-74ff55c5b-p8r76
748470c6fcbdd ba04bb24b9575 8 minutes ago Exited storage-provisioner 0 05416f9ca5341 storage-provisioner
63569e363dfc9 4740c1948d3fc 8 minutes ago Exited kindnet-cni 0 e19e7cf5c26e6 kindnet-697g5
2e49139787db6 25a5233254979 8 minutes ago Exited kube-proxy 0 d7ce57cfecff4 kube-proxy-9tnq4
35b9ec8e07a28 e7605f88f17d6 9 minutes ago Exited kube-scheduler 0 2dc8e02860235 kube-scheduler-old-k8s-version-152774
1fa6553e277e6 1df8a2b116bd1 9 minutes ago Exited kube-controller-manager 0 6b383eaa7303f kube-controller-manager-old-k8s-version-152774
2b9fb6b94c99c 2c08bbbc02d3a 9 minutes ago Exited kube-apiserver 0 f1c7811254f5d kube-apiserver-old-k8s-version-152774
2836d24b2e217 05b738aa1bc63 9 minutes ago Exited etcd 0 0acd73641ceab etcd-old-k8s-version-152774
==> containerd <==
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.477746703Z" level=info msg="CreateContainer within sandbox \"1dd65b48d5af67efe61e69568b438eaf7d034fe948a9230855def7a52542249b\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:4,} returns container id \"21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32\""
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.478483420Z" level=info msg="StartContainer for \"21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32\""
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.549569034Z" level=info msg="StartContainer for \"21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32\" returns successfully"
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.573692985Z" level=info msg="shim disconnected" id=21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.573764762Z" level=warning msg="cleaning up after shim disconnected" id=21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32 namespace=k8s.io
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.573776880Z" level=info msg="cleaning up dead shim"
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.585221389Z" level=warning msg="cleanup warnings time=\"2024-03-27T19:16:55Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=2940 runtime=io.containerd.runc.v2\n"
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.935735724Z" level=info msg="RemoveContainer for \"1a7f3a2be6d66bac1bd4278acf7a8eb272ad4f8839c651d6f46e6f0efe9e720a\""
Mar 27 19:16:55 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:16:55.948229663Z" level=info msg="RemoveContainer for \"1a7f3a2be6d66bac1bd4278acf7a8eb272ad4f8839c651d6f46e6f0efe9e720a\" returns successfully"
Mar 27 19:17:40 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:17:40.448457040Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:17:40 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:17:40.454131329Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Mar 27 19:17:40 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:17:40.455804758Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.450317362Z" level=info msg="CreateContainer within sandbox \"1dd65b48d5af67efe61e69568b438eaf7d034fe948a9230855def7a52542249b\" for container &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,}"
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.469706605Z" level=info msg="CreateContainer within sandbox \"1dd65b48d5af67efe61e69568b438eaf7d034fe948a9230855def7a52542249b\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,} returns container id \"a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d\""
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.470848972Z" level=info msg="StartContainer for \"a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d\""
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.535576348Z" level=info msg="StartContainer for \"a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d\" returns successfully"
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.561204935Z" level=info msg="shim disconnected" id=a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.561267998Z" level=warning msg="cleaning up after shim disconnected" id=a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d namespace=k8s.io
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.561281503Z" level=info msg="cleaning up dead shim"
Mar 27 19:18:24 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:24.570846299Z" level=warning msg="cleanup warnings time=\"2024-03-27T19:18:24Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=3195 runtime=io.containerd.runc.v2\n"
Mar 27 19:18:25 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:25.178651233Z" level=info msg="RemoveContainer for \"21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32\""
Mar 27 19:18:25 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:18:25.185522830Z" level=info msg="RemoveContainer for \"21c9ea9a959b8f3b6a66832830fce902c8d962cd02e6469ea66baad7980f7f32\" returns successfully"
Mar 27 19:20:30 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:20:30.449641525Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:30 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:20:30.457730518Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Mar 27 19:20:30 old-k8s-version-152774 containerd[569]: time="2024-03-27T19:20:30.462533893Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
==> coredns [1b4d47ebc9130e8cd1b74e892beda83c30f1f258580f79b8f3190ee7a611bf8b] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:57478 - 41977 "HINFO IN 3337084615773920932.5078315597824381027. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.033193806s
==> coredns [6befe5bb1170851080229896575b31f50dd1e0c6cb91fc60734e6462adb9b228] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:49400 - 4941 "HINFO IN 8324483486139086489.7153863531144397180. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.012830981s
==> describe nodes <==
Name: old-k8s-version-152774
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-152774
kubernetes.io/os=linux
minikube.k8s.io/commit=475b39f6a1dc94a0c7060d2eec10d9b995edcd28
minikube.k8s.io/name=old-k8s-version-152774
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_03_27T19_11_49_0700
minikube.k8s.io/version=v1.33.0-beta.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 27 Mar 2024 19:11:45 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-152774
AcquireTime: <unset>
RenewTime: Wed, 27 Mar 2024 19:20:34 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Wed, 27 Mar 2024 19:15:52 +0000 Wed, 27 Mar 2024 19:11:38 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Wed, 27 Mar 2024 19:15:52 +0000 Wed, 27 Mar 2024 19:11:38 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Wed, 27 Mar 2024 19:15:52 +0000 Wed, 27 Mar 2024 19:11:38 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Wed, 27 Mar 2024 19:15:52 +0000 Wed, 27 Mar 2024 19:12:03 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-152774
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
System Info:
Machine ID: 08663e09db0d47ee825abb8dd5cf8ac3
System UUID: 112fef1d-d9f5-44a5-aaa6-bafff3bceaaa
Boot ID: 3ced2ab6-f576-451e-8762-49421fd13f89
Kernel Version: 5.15.0-1056-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.6.28
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m37s
kube-system coredns-74ff55c5b-p8r76 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m35s
kube-system etcd-old-k8s-version-152774 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m42s
kube-system kindnet-697g5 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m36s
kube-system kube-apiserver-old-k8s-version-152774 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m42s
kube-system kube-controller-manager-old-k8s-version-152774 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m42s
kube-system kube-proxy-9tnq4 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m36s
kube-system kube-scheduler-old-k8s-version-152774 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m42s
kube-system metrics-server-9975d5f86-p8k5b 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m26s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m34s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-9ddc4 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m30s
kubernetes-dashboard kubernetes-dashboard-cd95d586-cchqs 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m30s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 9m2s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 9m2s (x4 over 9m2s) kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 9m2s (x4 over 9m2s) kubelet Node old-k8s-version-152774 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 9m2s (x4 over 9m2s) kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 9m2s kubelet Updated Node Allocatable limit across pods
Normal Starting 8m43s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m43s kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m43s kubelet Node old-k8s-version-152774 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m43s kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m42s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m36s kubelet Node old-k8s-version-152774 status is now: NodeReady
Normal Starting 8m35s kube-proxy Starting kube-proxy.
Normal Starting 5m57s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m57s (x8 over 5m57s) kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m57s (x7 over 5m57s) kubelet Node old-k8s-version-152774 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m57s (x8 over 5m57s) kubelet Node old-k8s-version-152774 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m57s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m45s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.000759] FS-Cache: N-cookie c=0000001f [p=00000015 fl=2 nc=0 na=1]
[ +0.001048] FS-Cache: N-cookie d=00000000b6bf5524{9p.inode} n=000000000e01dd23
[ +0.001112] FS-Cache: N-key=[8] '0c6fed0000000000'
[ +2.863158] FS-Cache: Duplicate cookie detected
[ +0.000707] FS-Cache: O-cookie c=00000016 [p=00000015 fl=226 nc=0 na=1]
[ +0.000978] FS-Cache: O-cookie d=00000000b6bf5524{9p.inode} n=00000000622d59e2
[ +0.001040] FS-Cache: O-key=[8] '0b6fed0000000000'
[ +0.000739] FS-Cache: N-cookie c=00000021 [p=00000015 fl=2 nc=0 na=1]
[ +0.000923] FS-Cache: N-cookie d=00000000b6bf5524{9p.inode} n=000000007503e9b8
[ +0.001184] FS-Cache: N-key=[8] '0b6fed0000000000'
[ +0.278616] FS-Cache: Duplicate cookie detected
[ +0.000731] FS-Cache: O-cookie c=0000001b [p=00000015 fl=226 nc=0 na=1]
[ +0.001050] FS-Cache: O-cookie d=00000000b6bf5524{9p.inode} n=00000000ade91ad5
[ +0.001135] FS-Cache: O-key=[8] '116fed0000000000'
[ +0.000789] FS-Cache: N-cookie c=00000022 [p=00000015 fl=2 nc=0 na=1]
[ +0.000923] FS-Cache: N-cookie d=00000000b6bf5524{9p.inode} n=0000000033515bb1
[ +0.001076] FS-Cache: N-key=[8] '116fed0000000000'
[ +3.283918] FS-Cache: Duplicate cookie detected
[ +0.000733] FS-Cache: O-cookie c=00000025 [p=00000002 fl=222 nc=0 na=1]
[ +0.001054] FS-Cache: O-cookie d=00000000940205da{9P.session} n=00000000f13d3b57
[ +0.001072] FS-Cache: O-key=[10] '34323936393931353332'
[ +0.000755] FS-Cache: N-cookie c=00000026 [p=00000002 fl=2 nc=0 na=1]
[ +0.000967] FS-Cache: N-cookie d=00000000940205da{9P.session} n=0000000099aef1c6
[ +0.001172] FS-Cache: N-key=[10] '34323936393931353332'
[Mar27 19:08] overlayfs: failed to resolve '/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/49/fs': -2
==> etcd [2836d24b2e21711e774643cc71660b139c080043c12085751d617eb1845d3a42] <==
raft2024/03/27 19:11:38 INFO: 9f0758e1c58a86ed received MsgVoteResp from 9f0758e1c58a86ed at term 2
raft2024/03/27 19:11:38 INFO: 9f0758e1c58a86ed became leader at term 2
raft2024/03/27 19:11:38 INFO: raft.node: 9f0758e1c58a86ed elected leader 9f0758e1c58a86ed at term 2
2024-03-27 19:11:38.939632 I | etcdserver: published {Name:old-k8s-version-152774 ClientURLs:[https://192.168.85.2:2379]} to cluster 68eaea490fab4e05
2024-03-27 19:11:38.939800 I | embed: ready to serve client requests
2024-03-27 19:11:38.941736 I | embed: serving client requests on 192.168.85.2:2379
2024-03-27 19:11:38.942124 I | embed: ready to serve client requests
2024-03-27 19:11:38.942341 I | etcdserver: setting up the initial cluster version to 3.4
2024-03-27 19:11:38.954349 N | etcdserver/membership: set the initial cluster version to 3.4
2024-03-27 19:11:38.954632 I | etcdserver/api: enabled capabilities for version 3.4
2024-03-27 19:11:38.960630 I | embed: serving client requests on 127.0.0.1:2379
2024-03-27 19:12:04.962024 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:07.893620 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:17.893553 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:27.893715 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:37.893712 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:47.893532 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:12:57.893585 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:07.893734 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:17.893636 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:27.894733 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:37.899105 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:47.893702 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:13:57.893809 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:14:07.893710 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [52e0ba1f0d529b25f17986e7bb6a992b9b1ed9ecf6b8389d60e2d4ce42ef57fb] <==
2024-03-27 19:16:30.523303 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:16:40.523179 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:16:50.523127 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:00.532518 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:10.523085 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:20.523271 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:30.523146 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:40.523210 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:17:50.523322 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:00.523220 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:10.523118 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:20.523188 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:30.523127 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:40.523301 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:18:50.523395 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:00.523349 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:10.523217 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:20.523201 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:30.525278 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:40.523345 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:19:50.523135 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:20:00.534861 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:20:10.523985 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:20:20.523610 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-27 19:20:30.524022 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
19:20:39 up 3:02, 0 users, load average: 1.67, 1.94, 2.41
Linux old-k8s-version-152774 5.15.0-1056-aws #61~20.04.1-Ubuntu SMP Wed Mar 13 17:45:04 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [207572546e278487abb6f6e042c46965634a924b922e4ccd98a323b846424910] <==
I0327 19:18:36.550727 1 main.go:227] handling current node
I0327 19:18:46.566298 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:18:46.566330 1 main.go:227] handling current node
I0327 19:18:56.571266 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:18:56.571826 1 main.go:227] handling current node
I0327 19:19:06.588451 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:06.588705 1 main.go:227] handling current node
I0327 19:19:16.603443 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:16.603479 1 main.go:227] handling current node
I0327 19:19:26.618571 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:26.618600 1 main.go:227] handling current node
I0327 19:19:36.634445 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:36.634477 1 main.go:227] handling current node
I0327 19:19:46.653584 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:46.653614 1 main.go:227] handling current node
I0327 19:19:56.667368 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:19:56.667578 1 main.go:227] handling current node
I0327 19:20:06.681571 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:20:06.681816 1 main.go:227] handling current node
I0327 19:20:16.702245 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:20:16.702591 1 main.go:227] handling current node
I0327 19:20:26.713675 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:20:26.713706 1 main.go:227] handling current node
I0327 19:20:36.728890 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:20:36.728921 1 main.go:227] handling current node
==> kindnet [63569e363dfc9296fa9ff768788b490ad6ed9f778249d11b66de849c6dcad737] <==
podIP = 192.168.85.2
I0327 19:12:04.745646 1 main.go:116] setting mtu 1500 for CNI
I0327 19:12:04.745680 1 main.go:146] kindnetd IP family: "ipv4"
I0327 19:12:04.745693 1 main.go:150] noMask IPv4 subnets: [10.244.0.0/16]
I0327 19:12:34.963507 1 main.go:191] Failed to get nodes, retrying after error: Get "https://10.96.0.1:443/api/v1/nodes": dial tcp 10.96.0.1:443: i/o timeout
I0327 19:12:34.977610 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:12:34.977647 1 main.go:227] handling current node
I0327 19:12:44.995953 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:12:44.995985 1 main.go:227] handling current node
I0327 19:12:55.027553 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:12:55.027585 1 main.go:227] handling current node
I0327 19:13:05.037353 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:05.037389 1 main.go:227] handling current node
I0327 19:13:15.067178 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:15.067209 1 main.go:227] handling current node
I0327 19:13:25.095556 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:25.095589 1 main.go:227] handling current node
I0327 19:13:35.106039 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:35.106079 1 main.go:227] handling current node
I0327 19:13:45.126709 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:45.126748 1 main.go:227] handling current node
I0327 19:13:55.153817 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:13:55.153853 1 main.go:227] handling current node
I0327 19:14:05.168826 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0327 19:14:05.169072 1 main.go:227] handling current node
==> kube-apiserver [0ffe2baaed51274b240657c18c34ed424b18570b32a5b170fdc9ee24c9784f68] <==
I0327 19:17:24.018866 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:17:24.018876 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0327 19:17:54.211463 1 handler_proxy.go:102] no RequestInfo found in the context
E0327 19:17:54.211532 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0327 19:17:54.211540 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0327 19:18:07.518926 1 client.go:360] parsed scheme: "passthrough"
I0327 19:18:07.518978 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:18:07.518988 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0327 19:18:38.990457 1 client.go:360] parsed scheme: "passthrough"
I0327 19:18:38.990503 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:18:38.990512 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0327 19:19:14.769108 1 client.go:360] parsed scheme: "passthrough"
I0327 19:19:14.769163 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:19:14.769173 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0327 19:19:50.846508 1 client.go:360] parsed scheme: "passthrough"
I0327 19:19:50.846773 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:19:50.846862 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0327 19:19:52.728894 1 handler_proxy.go:102] no RequestInfo found in the context
E0327 19:19:52.729049 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0327 19:19:52.729082 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0327 19:20:33.869359 1 client.go:360] parsed scheme: "passthrough"
I0327 19:20:33.869406 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:20:33.869415 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [2b9fb6b94c99ccab1661b66d26ec6e79310f10d675d9070ea6be0268e91df23c] <==
I0327 19:11:46.114531 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0327 19:11:46.592026 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0327 19:11:46.632209 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0327 19:11:46.755459 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.85.2]
I0327 19:11:46.756680 1 controller.go:606] quota admission added evaluator for: endpoints
I0327 19:11:46.760589 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0327 19:11:47.850015 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0327 19:11:48.487244 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0327 19:11:48.631546 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0327 19:11:56.931384 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0327 19:12:03.755631 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0327 19:12:04.000898 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0327 19:12:21.086221 1 client.go:360] parsed scheme: "passthrough"
I0327 19:12:21.086281 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:12:21.086290 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0327 19:12:57.264424 1 client.go:360] parsed scheme: "passthrough"
I0327 19:12:57.264465 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:12:57.264473 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0327 19:13:34.609041 1 client.go:360] parsed scheme: "passthrough"
I0327 19:13:34.609175 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:13:34.609205 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
E0327 19:14:13.015797 1 available_controller.go:508] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
I0327 19:14:13.499347 1 client.go:360] parsed scheme: "passthrough"
I0327 19:14:13.499401 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0327 19:14:13.499411 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [1fa6553e277e60c4a15187316ea492539b4f45ccb37e8e5817ed466be9e7ad3b] <==
I0327 19:12:03.895312 1 disruption.go:339] Sending events to api server.
I0327 19:12:03.899268 1 shared_informer.go:247] Caches are synced for deployment
I0327 19:12:03.900178 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-legacy-unknown
I0327 19:12:03.900477 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-kubelet-serving
I0327 19:12:03.900495 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-kubelet-client
I0327 19:12:03.900543 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-kube-apiserver-client
I0327 19:12:03.909066 1 shared_informer.go:247] Caches are synced for certificate-csrapproving
I0327 19:12:03.932661 1 shared_informer.go:247] Caches are synced for resource quota
I0327 19:12:03.955010 1 shared_informer.go:247] Caches are synced for resource quota
I0327 19:12:04.062107 1 shared_informer.go:247] Caches are synced for attach detach
I0327 19:12:04.080534 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0327 19:12:04.125197 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-nrwsz"
E0327 19:12:04.147782 1 clusterroleaggregation_controller.go:181] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
I0327 19:12:04.232825 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0327 19:12:04.250507 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-p8r76"
I0327 19:12:04.433061 1 shared_informer.go:247] Caches are synced for garbage collector
I0327 19:12:04.508976 1 shared_informer.go:247] Caches are synced for garbage collector
I0327 19:12:04.509000 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0327 19:12:05.757486 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0327 19:12:05.776554 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-nrwsz"
I0327 19:12:08.837242 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0327 19:14:12.692140 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0327 19:14:12.844780 1 clusterroleaggregation_controller.go:181] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
E0327 19:14:12.845019 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
E0327 19:14:12.870870 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
==> kube-controller-manager [6c75c3cfda9f5d472ea71bc2fbc3f0ab362fba5a1893fea084b75b36c818698f] <==
W0327 19:16:14.875292 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:16:40.921258 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:16:46.525853 1 request.go:655] Throttling request took 1.048225425s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0327 19:16:47.377519 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:17:11.423131 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:17:19.027957 1 request.go:655] Throttling request took 1.048391446s, request: GET:https://192.168.85.2:8443/apis/batch/v1beta1?timeout=32s
W0327 19:17:19.879475 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:17:41.925102 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:17:51.530110 1 request.go:655] Throttling request took 1.04847049s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0327 19:17:52.381436 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:18:12.431392 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:18:24.031823 1 request.go:655] Throttling request took 1.047670315s, request: GET:https://192.168.85.2:8443/apis/autoscaling/v1?timeout=32s
W0327 19:18:24.883281 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:18:42.933166 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:18:56.533641 1 request.go:655] Throttling request took 1.046178445s, request: GET:https://192.168.85.2:8443/apis/policy/v1beta1?timeout=32s
W0327 19:18:57.385000 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:19:13.435010 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:19:29.035429 1 request.go:655] Throttling request took 1.04836813s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1?timeout=32s
W0327 19:19:29.887151 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:19:43.936903 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:20:01.537510 1 request.go:655] Throttling request took 1.039136638s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0327 19:20:02.389345 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0327 19:20:14.438445 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0327 19:20:34.039880 1 request.go:655] Throttling request took 1.048189952s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0327 19:20:34.891257 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [2e49139787db659cffda7b1b5380735bd4a82e157bb541023d540ad247fc9f83] <==
I0327 19:12:04.830767 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0327 19:12:04.830867 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0327 19:12:04.903014 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0327 19:12:04.903106 1 server_others.go:185] Using iptables Proxier.
I0327 19:12:04.903307 1 server.go:650] Version: v1.20.0
I0327 19:12:04.903775 1 config.go:315] Starting service config controller
I0327 19:12:04.903792 1 shared_informer.go:240] Waiting for caches to sync for service config
I0327 19:12:04.906124 1 config.go:224] Starting endpoint slice config controller
I0327 19:12:04.906153 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0327 19:12:05.003922 1 shared_informer.go:247] Caches are synced for service config
I0327 19:12:05.006275 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [de25424b8f929fa617fe73fa2d1e6b021b8f6a2f381c5b1327a3bddeddcafba5] <==
I0327 19:14:54.067737 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0327 19:14:54.067898 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0327 19:14:54.087360 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0327 19:14:54.087590 1 server_others.go:185] Using iptables Proxier.
I0327 19:14:54.087970 1 server.go:650] Version: v1.20.0
I0327 19:14:54.090766 1 config.go:315] Starting service config controller
I0327 19:14:54.090808 1 shared_informer.go:240] Waiting for caches to sync for service config
I0327 19:14:54.091166 1 config.go:224] Starting endpoint slice config controller
I0327 19:14:54.091190 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0327 19:14:54.193674 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0327 19:14:54.193764 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [35b9ec8e07a2859078518707aa897230e2d986371cc4c0fcc9de59441501d424] <==
I0327 19:11:41.064585 1 serving.go:331] Generated self-signed cert in-memory
W0327 19:11:45.293129 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0327 19:11:45.293211 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0327 19:11:45.293393 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0327 19:11:45.293409 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0327 19:11:45.421459 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0327 19:11:45.421595 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0327 19:11:45.421752 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0327 19:11:45.421776 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E0327 19:11:45.426355 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0327 19:11:45.436076 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0327 19:11:45.436178 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0327 19:11:45.437156 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0327 19:11:45.437400 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0327 19:11:45.437559 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0327 19:11:45.437782 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0327 19:11:45.438025 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0327 19:11:45.438648 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0327 19:11:45.438836 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0327 19:11:45.439005 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0327 19:11:45.439119 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0327 19:11:46.275256 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0327 19:11:46.275603 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0327 19:11:46.334992 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
I0327 19:11:46.721867 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [45f861afe1590718715920ce8ac726bfbc067e6e7556bc6b43bf587308954d6e] <==
I0327 19:14:47.399314 1 serving.go:331] Generated self-signed cert in-memory
W0327 19:14:51.690521 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0327 19:14:51.690558 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0327 19:14:51.690566 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0327 19:14:51.690572 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0327 19:14:51.875897 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0327 19:14:51.875994 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0327 19:14:51.876010 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0327 19:14:51.876025 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0327 19:14:52.086227 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Mar 27 19:18:55 old-k8s-version-152774 kubelet[662]: E0327 19:18:55.447433 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:08 old-k8s-version-152774 kubelet[662]: E0327 19:19:08.447831 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: I0327 19:19:09.447187 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:19:09 old-k8s-version-152774 kubelet[662]: E0327 19:19:09.447537 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:20 old-k8s-version-152774 kubelet[662]: E0327 19:19:20.448039 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: I0327 19:19:21.447180 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:19:21 old-k8s-version-152774 kubelet[662]: E0327 19:19:21.447551 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: I0327 19:19:32.447848 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:19:32 old-k8s-version-152774 kubelet[662]: E0327 19:19:32.448689 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:34 old-k8s-version-152774 kubelet[662]: E0327 19:19:34.449620 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: I0327 19:19:47.447771 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448095 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:19:47 old-k8s-version-152774 kubelet[662]: E0327 19:19:47.448431 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.447877 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: I0327 19:20:01.449276 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:20:01 old-k8s-version-152774 kubelet[662]: E0327 19:20:01.449776 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: I0327 19:20:15.447145 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.448719 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 27 19:20:15 old-k8s-version-152774 kubelet[662]: E0327 19:20:15.449702 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:29 old-k8s-version-152774 kubelet[662]: I0327 19:20:29.447173 662 scope.go:95] [topologymanager] RemoveContainer - Container ID: a4f7bac13f4a7ad1b325238364476c851a38d59839ef5df236e378f2733c412d
Mar 27 19:20:29 old-k8s-version-152774 kubelet[662]: E0327 19:20:29.447515 662 pod_workers.go:191] Error syncing pod a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45 ("dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-9ddc4_kubernetes-dashboard(a0bbd7ce-44b0-41f8-a72b-f3cb4d154a45)"
Mar 27 19:20:30 old-k8s-version-152774 kubelet[662]: E0327 19:20:30.462794 662 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 27 19:20:30 old-k8s-version-152774 kubelet[662]: E0327 19:20:30.462846 662 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 27 19:20:30 old-k8s-version-152774 kubelet[662]: E0327 19:20:30.462976 662 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-bntmf,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-p8k5b_kube-system(59b3461
e-c3ed-454d-a8ee-09516ea76215): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Mar 27 19:20:30 old-k8s-version-152774 kubelet[662]: E0327 19:20:30.463011 662 pod_workers.go:191] Error syncing pod 59b3461e-c3ed-454d-a8ee-09516ea76215 ("metrics-server-9975d5f86-p8k5b_kube-system(59b3461e-c3ed-454d-a8ee-09516ea76215)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
==> kubernetes-dashboard [44d7fd2c3587be7277228efcfc0f30d5bebe1e600aa53bca216baac927c90585] <==
2024/03/27 19:15:14 Using namespace: kubernetes-dashboard
2024/03/27 19:15:14 Using in-cluster config to connect to apiserver
2024/03/27 19:15:14 Using secret token for csrf signing
2024/03/27 19:15:14 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/03/27 19:15:14 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/03/27 19:15:14 Successful initial request to the apiserver, version: v1.20.0
2024/03/27 19:15:14 Generating JWE encryption key
2024/03/27 19:15:14 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/03/27 19:15:14 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/03/27 19:15:14 Initializing JWE encryption key from synchronized object
2024/03/27 19:15:14 Creating in-cluster Sidecar client
2024/03/27 19:15:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:15:14 Serving insecurely on HTTP port: 9090
2024/03/27 19:15:44 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:16:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:16:44 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:17:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:17:44 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:18:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:18:44 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:19:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:19:44 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:20:14 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/27 19:15:14 Starting overwatch
==> storage-provisioner [63d5d568691f6db6a790589498e8a601d0cb73ecb855c686a3ae494938f83a07] <==
I0327 19:14:55.602388 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0327 19:14:55.620743 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0327 19:14:55.620985 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0327 19:15:13.084792 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0327 19:15:13.085732 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-152774_a1db7022-fb20-4b57-82e6-5501025ce318!
I0327 19:15:13.086318 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"49b38ac5-5af7-42e8-aa05-1b36cfcbc6b8", APIVersion:"v1", ResourceVersion:"806", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-152774_a1db7022-fb20-4b57-82e6-5501025ce318 became leader
I0327 19:15:13.186177 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-152774_a1db7022-fb20-4b57-82e6-5501025ce318!
==> storage-provisioner [748470c6fcbdd635075dc2f53e0b1b149e07e01ca141c6f054d52eb75c80d448] <==
I0327 19:12:06.451355 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0327 19:12:06.469074 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0327 19:12:06.469403 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0327 19:12:06.480630 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"49b38ac5-5af7-42e8-aa05-1b36cfcbc6b8", APIVersion:"v1", ResourceVersion:"477", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-152774_784fbcb4-a082-42e2-b722-8df1e1695e63 became leader
I0327 19:12:06.480192 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0327 19:12:06.481025 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-152774_784fbcb4-a082-42e2-b722-8df1e1695e63!
I0327 19:12:06.581909 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-152774_784fbcb4-a082-42e2-b722-8df1e1695e63!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-152774 -n old-k8s-version-152774
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-152774 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-p8k5b
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-152774 describe pod metrics-server-9975d5f86-p8k5b
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-152774 describe pod metrics-server-9975d5f86-p8k5b: exit status 1 (141.899724ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-p8k5b" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-152774 describe pod metrics-server-9975d5f86-p8k5b: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (375.26s)