=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-591842 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-591842 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m10.305554022s)
-- stdout --
* [old-k8s-version-591842] minikube v1.32.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18213
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18213-3295134/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18213-3295134/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.28.4 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.28.4
* Using the docker driver based on existing profile
* Starting "old-k8s-version-591842" primary control-plane node in "old-k8s-version-591842" cluster
* Pulling base image v0.0.42-1710284843-18375 ...
* Restarting existing docker container for "old-k8s-version-591842" ...
* Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-591842 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
-- /stdout --
** stderr **
I0315 07:45:31.788037 3490722 out.go:291] Setting OutFile to fd 1 ...
I0315 07:45:31.788242 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:45:31.788267 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:45:31.788285 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:45:31.788590 3490722 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18213-3295134/.minikube/bin
I0315 07:45:31.788989 3490722 out.go:298] Setting JSON to false
I0315 07:45:31.790118 3490722 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":59276,"bootTime":1710429456,"procs":211,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I0315 07:45:31.790209 3490722 start.go:139] virtualization:
I0315 07:45:31.793143 3490722 out.go:177] * [old-k8s-version-591842] minikube v1.32.0 on Ubuntu 20.04 (arm64)
I0315 07:45:31.796058 3490722 out.go:177] - MINIKUBE_LOCATION=18213
I0315 07:45:31.797898 3490722 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0315 07:45:31.796204 3490722 notify.go:220] Checking for updates...
I0315 07:45:31.802946 3490722 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18213-3295134/kubeconfig
I0315 07:45:31.805947 3490722 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18213-3295134/.minikube
I0315 07:45:31.808541 3490722 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0315 07:45:31.810656 3490722 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0315 07:45:31.813566 3490722 config.go:182] Loaded profile config "old-k8s-version-591842": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0315 07:45:31.816469 3490722 out.go:177] * Kubernetes 1.28.4 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.28.4
I0315 07:45:31.818840 3490722 driver.go:392] Setting default libvirt URI to qemu:///system
I0315 07:45:31.849482 3490722 docker.go:122] docker version: linux-25.0.4:Docker Engine - Community
I0315 07:45:31.849644 3490722 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0315 07:45:31.903126 3490722 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:42 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-15 07:45:31.894226267 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0315 07:45:31.903229 3490722 docker.go:295] overlay module found
I0315 07:45:31.905731 3490722 out.go:177] * Using the docker driver based on existing profile
I0315 07:45:31.908180 3490722 start.go:297] selected driver: docker
I0315 07:45:31.908210 3490722 start.go:901] validating driver "docker" against &{Name:old-k8s-version-591842 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-591842 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0315 07:45:31.908319 3490722 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0315 07:45:31.908927 3490722 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0315 07:45:31.960943 3490722 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:42 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-15 07:45:31.951244949 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0315 07:45:31.961283 3490722 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0315 07:45:31.961354 3490722 cni.go:84] Creating CNI manager for ""
I0315 07:45:31.961373 3490722 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0315 07:45:31.961424 3490722 start.go:340] cluster config:
{Name:old-k8s-version-591842 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-591842 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0315 07:45:31.969332 3490722 out.go:177] * Starting "old-k8s-version-591842" primary control-plane node in "old-k8s-version-591842" cluster
I0315 07:45:31.971719 3490722 cache.go:121] Beginning downloading kic base image for docker with containerd
I0315 07:45:31.973881 3490722 out.go:177] * Pulling base image v0.0.42-1710284843-18375 ...
I0315 07:45:31.976061 3490722 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0315 07:45:31.976110 3490722 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0315 07:45:31.976130 3490722 cache.go:56] Caching tarball of preloaded images
I0315 07:45:31.976215 3490722 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f in local docker daemon
I0315 07:45:31.976232 3490722 preload.go:173] Found /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0315 07:45:31.976242 3490722 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0315 07:45:31.976355 3490722 profile.go:142] Saving config to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/config.json ...
I0315 07:45:31.991744 3490722 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f in local docker daemon, skipping pull
I0315 07:45:31.991769 3490722 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f exists in daemon, skipping load
I0315 07:45:31.991790 3490722 cache.go:194] Successfully downloaded all kic artifacts
I0315 07:45:31.991831 3490722 start.go:360] acquireMachinesLock for old-k8s-version-591842: {Name:mk63b04c4dd89051346b19e9171d31d6d188f42c Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0315 07:45:31.991906 3490722 start.go:364] duration metric: took 49.287µs to acquireMachinesLock for "old-k8s-version-591842"
I0315 07:45:31.991933 3490722 start.go:96] Skipping create...Using existing machine configuration
I0315 07:45:31.991939 3490722 fix.go:54] fixHost starting:
I0315 07:45:31.992200 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:32.015340 3490722 fix.go:112] recreateIfNeeded on old-k8s-version-591842: state=Stopped err=<nil>
W0315 07:45:32.015377 3490722 fix.go:138] unexpected machine state, will restart: <nil>
I0315 07:45:32.018244 3490722 out.go:177] * Restarting existing docker container for "old-k8s-version-591842" ...
I0315 07:45:32.020480 3490722 cli_runner.go:164] Run: docker start old-k8s-version-591842
I0315 07:45:32.369154 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:32.391641 3490722 kic.go:430] container "old-k8s-version-591842" state is running.
I0315 07:45:32.392049 3490722 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-591842
I0315 07:45:32.413426 3490722 profile.go:142] Saving config to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/config.json ...
I0315 07:45:32.413654 3490722 machine.go:94] provisionDockerMachine start ...
I0315 07:45:32.413707 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:32.435169 3490722 main.go:141] libmachine: Using SSH client type: native
I0315 07:45:32.435451 3490722 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36975 <nil> <nil>}
I0315 07:45:32.435463 3490722 main.go:141] libmachine: About to run SSH command:
hostname
I0315 07:45:32.436003 3490722 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:54058->127.0.0.1:36975: read: connection reset by peer
I0315 07:45:35.582619 3490722 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-591842
I0315 07:45:35.582649 3490722 ubuntu.go:169] provisioning hostname "old-k8s-version-591842"
I0315 07:45:35.582715 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:35.604496 3490722 main.go:141] libmachine: Using SSH client type: native
I0315 07:45:35.604750 3490722 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36975 <nil> <nil>}
I0315 07:45:35.604767 3490722 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-591842 && echo "old-k8s-version-591842" | sudo tee /etc/hostname
I0315 07:45:35.764476 3490722 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-591842
I0315 07:45:35.764557 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:35.782515 3490722 main.go:141] libmachine: Using SSH client type: native
I0315 07:45:35.782767 3490722 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36975 <nil> <nil>}
I0315 07:45:35.782791 3490722 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-591842' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-591842/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-591842' | sudo tee -a /etc/hosts;
fi
fi
I0315 07:45:35.927182 3490722 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0315 07:45:35.927210 3490722 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18213-3295134/.minikube CaCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18213-3295134/.minikube}
I0315 07:45:35.927235 3490722 ubuntu.go:177] setting up certificates
I0315 07:45:35.927244 3490722 provision.go:84] configureAuth start
I0315 07:45:35.927308 3490722 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-591842
I0315 07:45:35.947333 3490722 provision.go:143] copyHostCerts
I0315 07:45:35.947410 3490722 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem, removing ...
I0315 07:45:35.947431 3490722 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem
I0315 07:45:35.947505 3490722 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem (1078 bytes)
I0315 07:45:35.947607 3490722 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem, removing ...
I0315 07:45:35.947618 3490722 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem
I0315 07:45:35.947646 3490722 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem (1123 bytes)
I0315 07:45:35.947708 3490722 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem, removing ...
I0315 07:45:35.947716 3490722 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem
I0315 07:45:35.947743 3490722 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem (1679 bytes)
I0315 07:45:35.947800 3490722 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-591842 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-591842]
I0315 07:45:36.732700 3490722 provision.go:177] copyRemoteCerts
I0315 07:45:36.732773 3490722 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0315 07:45:36.732819 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:36.750039 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:36.852024 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0315 07:45:36.879033 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0315 07:45:36.903402 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0315 07:45:36.929005 3490722 provision.go:87] duration metric: took 1.001725871s to configureAuth
I0315 07:45:36.929037 3490722 ubuntu.go:193] setting minikube options for container-runtime
I0315 07:45:36.929229 3490722 config.go:182] Loaded profile config "old-k8s-version-591842": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0315 07:45:36.929240 3490722 machine.go:97] duration metric: took 4.515578003s to provisionDockerMachine
I0315 07:45:36.929248 3490722 start.go:293] postStartSetup for "old-k8s-version-591842" (driver="docker")
I0315 07:45:36.929258 3490722 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0315 07:45:36.929319 3490722 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0315 07:45:36.929374 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:36.947821 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:37.053905 3490722 ssh_runner.go:195] Run: cat /etc/os-release
I0315 07:45:37.057162 3490722 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0315 07:45:37.057200 3490722 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0315 07:45:37.057211 3490722 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0315 07:45:37.057217 3490722 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0315 07:45:37.057227 3490722 filesync.go:126] Scanning /home/jenkins/minikube-integration/18213-3295134/.minikube/addons for local assets ...
I0315 07:45:37.057291 3490722 filesync.go:126] Scanning /home/jenkins/minikube-integration/18213-3295134/.minikube/files for local assets ...
I0315 07:45:37.057378 3490722 filesync.go:149] local asset: /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem -> 33005502.pem in /etc/ssl/certs
I0315 07:45:37.057528 3490722 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0315 07:45:37.067864 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem --> /etc/ssl/certs/33005502.pem (1708 bytes)
I0315 07:45:37.093029 3490722 start.go:296] duration metric: took 163.765321ms for postStartSetup
I0315 07:45:37.093115 3490722 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0315 07:45:37.093166 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:37.109205 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:37.205207 3490722 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0315 07:45:37.209662 3490722 fix.go:56] duration metric: took 5.217715301s for fixHost
I0315 07:45:37.209687 3490722 start.go:83] releasing machines lock for "old-k8s-version-591842", held for 5.217766064s
I0315 07:45:37.209773 3490722 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-591842
I0315 07:45:37.226805 3490722 ssh_runner.go:195] Run: cat /version.json
I0315 07:45:37.226855 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:37.227132 3490722 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0315 07:45:37.227279 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:37.244599 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:37.254851 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:37.342648 3490722 ssh_runner.go:195] Run: systemctl --version
I0315 07:45:37.499541 3490722 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0315 07:45:37.503980 3490722 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0315 07:45:37.523064 3490722 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0315 07:45:37.523199 3490722 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0315 07:45:37.533446 3490722 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0315 07:45:37.533470 3490722 start.go:494] detecting cgroup driver to use...
I0315 07:45:37.533522 3490722 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0315 07:45:37.533593 3490722 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0315 07:45:37.553752 3490722 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0315 07:45:37.568306 3490722 docker.go:217] disabling cri-docker service (if available) ...
I0315 07:45:37.568392 3490722 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0315 07:45:37.581701 3490722 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0315 07:45:37.593421 3490722 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0315 07:45:37.732633 3490722 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0315 07:45:37.856770 3490722 docker.go:233] disabling docker service ...
I0315 07:45:37.856886 3490722 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0315 07:45:37.871239 3490722 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0315 07:45:37.883022 3490722 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0315 07:45:37.983919 3490722 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0315 07:45:38.089870 3490722 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0315 07:45:38.105175 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0315 07:45:38.126181 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0315 07:45:38.140438 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0315 07:45:38.153274 3490722 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0315 07:45:38.153347 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0315 07:45:38.167814 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0315 07:45:38.179428 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0315 07:45:38.193508 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0315 07:45:38.204454 3490722 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0315 07:45:38.221203 3490722 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0315 07:45:38.235203 3490722 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0315 07:45:38.245306 3490722 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0315 07:45:38.254189 3490722 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0315 07:45:38.349954 3490722 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0315 07:45:38.528206 3490722 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0315 07:45:38.528306 3490722 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0315 07:45:38.534868 3490722 start.go:562] Will wait 60s for crictl version
I0315 07:45:38.534966 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:45:38.538856 3490722 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0315 07:45:38.589746 3490722 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0315 07:45:38.589862 3490722 ssh_runner.go:195] Run: containerd --version
I0315 07:45:38.612328 3490722 ssh_runner.go:195] Run: containerd --version
I0315 07:45:38.640217 3490722 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.6.28 ...
I0315 07:45:38.642327 3490722 cli_runner.go:164] Run: docker network inspect old-k8s-version-591842 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0315 07:45:38.663248 3490722 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0315 07:45:38.667822 3490722 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0315 07:45:38.680408 3490722 kubeadm.go:877] updating cluster {Name:old-k8s-version-591842 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-591842 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0315 07:45:38.680540 3490722 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0315 07:45:38.680599 3490722 ssh_runner.go:195] Run: sudo crictl images --output json
I0315 07:45:38.744940 3490722 containerd.go:612] all images are preloaded for containerd runtime.
I0315 07:45:38.744966 3490722 containerd.go:519] Images already preloaded, skipping extraction
I0315 07:45:38.745032 3490722 ssh_runner.go:195] Run: sudo crictl images --output json
I0315 07:45:38.787393 3490722 containerd.go:612] all images are preloaded for containerd runtime.
I0315 07:45:38.787417 3490722 cache_images.go:84] Images are preloaded, skipping loading
I0315 07:45:38.787426 3490722 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I0315 07:45:38.787543 3490722 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-591842 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-591842 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0315 07:45:38.787618 3490722 ssh_runner.go:195] Run: sudo crictl info
I0315 07:45:38.830926 3490722 cni.go:84] Creating CNI manager for ""
I0315 07:45:38.830950 3490722 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0315 07:45:38.830959 3490722 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0315 07:45:38.830981 3490722 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-591842 NodeName:old-k8s-version-591842 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0315 07:45:38.831146 3490722 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-591842"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0315 07:45:38.831224 3490722 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0315 07:45:38.840824 3490722 binaries.go:44] Found k8s binaries, skipping transfer
I0315 07:45:38.840902 3490722 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0315 07:45:38.850055 3490722 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0315 07:45:38.870210 3490722 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0315 07:45:38.888301 3490722 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0315 07:45:38.906789 3490722 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0315 07:45:38.910183 3490722 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0315 07:45:38.921449 3490722 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0315 07:45:39.008333 3490722 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0315 07:45:39.029971 3490722 certs.go:68] Setting up /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842 for IP: 192.168.76.2
I0315 07:45:39.030051 3490722 certs.go:194] generating shared ca certs ...
I0315 07:45:39.030107 3490722 certs.go:226] acquiring lock for ca certs: {Name:mk9abb58e338d3f021292a49b0c7ea22df42932a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:45:39.030288 3490722 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.key
I0315 07:45:39.030361 3490722 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.key
I0315 07:45:39.030400 3490722 certs.go:256] generating profile certs ...
I0315 07:45:39.030574 3490722 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/client.key
I0315 07:45:39.030696 3490722 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/apiserver.key.df0d5480
I0315 07:45:39.030778 3490722 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/proxy-client.key
I0315 07:45:39.030956 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550.pem (1338 bytes)
W0315 07:45:39.031016 3490722 certs.go:480] ignoring /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550_empty.pem, impossibly tiny 0 bytes
I0315 07:45:39.031042 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem (1675 bytes)
I0315 07:45:39.031156 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem (1078 bytes)
I0315 07:45:39.031216 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem (1123 bytes)
I0315 07:45:39.031277 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem (1679 bytes)
I0315 07:45:39.031394 3490722 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem (1708 bytes)
I0315 07:45:39.032291 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0315 07:45:39.063184 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0315 07:45:39.095832 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0315 07:45:39.147712 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0315 07:45:39.206776 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0315 07:45:39.255146 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0315 07:45:39.282312 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0315 07:45:39.307866 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/old-k8s-version-591842/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0315 07:45:39.357123 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem --> /usr/share/ca-certificates/33005502.pem (1708 bytes)
I0315 07:45:39.395381 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0315 07:45:39.422108 3490722 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550.pem --> /usr/share/ca-certificates/3300550.pem (1338 bytes)
I0315 07:45:39.452086 3490722 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0315 07:45:39.474930 3490722 ssh_runner.go:195] Run: openssl version
I0315 07:45:39.483243 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/33005502.pem && ln -fs /usr/share/ca-certificates/33005502.pem /etc/ssl/certs/33005502.pem"
I0315 07:45:39.498298 3490722 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/33005502.pem
I0315 07:45:39.502110 3490722 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 15 07:07 /usr/share/ca-certificates/33005502.pem
I0315 07:45:39.502177 3490722 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/33005502.pem
I0315 07:45:39.509810 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/33005502.pem /etc/ssl/certs/3ec20f2e.0"
I0315 07:45:39.520714 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0315 07:45:39.531541 3490722 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0315 07:45:39.535904 3490722 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 15 07:01 /usr/share/ca-certificates/minikubeCA.pem
I0315 07:45:39.535990 3490722 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0315 07:45:39.545789 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0315 07:45:39.555744 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3300550.pem && ln -fs /usr/share/ca-certificates/3300550.pem /etc/ssl/certs/3300550.pem"
I0315 07:45:39.565253 3490722 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3300550.pem
I0315 07:45:39.570419 3490722 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 15 07:07 /usr/share/ca-certificates/3300550.pem
I0315 07:45:39.570524 3490722 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3300550.pem
I0315 07:45:39.577981 3490722 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/3300550.pem /etc/ssl/certs/51391683.0"
I0315 07:45:39.586905 3490722 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0315 07:45:39.590804 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0315 07:45:39.598122 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0315 07:45:39.604865 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0315 07:45:39.611434 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0315 07:45:39.618501 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0315 07:45:39.630953 3490722 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0315 07:45:39.639138 3490722 kubeadm.go:391] StartCluster: {Name:old-k8s-version-591842 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-591842 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0315 07:45:39.639279 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0315 07:45:39.639366 3490722 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0315 07:45:39.694822 3490722 cri.go:89] found id: "f55d7b6eb2f971daafdac9c03f63d864ce34ba3d3c98bc693596d24c7eb631b5"
I0315 07:45:39.694889 3490722 cri.go:89] found id: "7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:45:39.694909 3490722 cri.go:89] found id: "1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:45:39.694937 3490722 cri.go:89] found id: "a957dd8724eabeb8d00b83d208e2da1998096377aba22d8b667ceb5b0e5bbfaa"
I0315 07:45:39.694957 3490722 cri.go:89] found id: "7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:45:39.694975 3490722 cri.go:89] found id: "6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:45:39.694994 3490722 cri.go:89] found id: "eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:45:39.695018 3490722 cri.go:89] found id: "aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:45:39.695036 3490722 cri.go:89] found id: "288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:45:39.695054 3490722 cri.go:89] found id: ""
I0315 07:45:39.695139 3490722 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0315 07:45:39.708696 3490722 cri.go:116] JSON = null
W0315 07:45:39.708802 3490722 kubeadm.go:398] unpause failed: list paused: list returned 0 containers, but ps returned 9
I0315 07:45:39.708893 3490722 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0315 07:45:39.719361 3490722 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0315 07:45:39.719384 3490722 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0315 07:45:39.719390 3490722 kubeadm.go:587] restartPrimaryControlPlane start ...
I0315 07:45:39.719439 3490722 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0315 07:45:39.729467 3490722 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0315 07:45:39.729905 3490722 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-591842" does not appear in /home/jenkins/minikube-integration/18213-3295134/kubeconfig
I0315 07:45:39.730015 3490722 kubeconfig.go:62] /home/jenkins/minikube-integration/18213-3295134/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-591842" cluster setting kubeconfig missing "old-k8s-version-591842" context setting]
I0315 07:45:39.730313 3490722 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/kubeconfig: {Name:mka8a8bb165c8233f51f8705aa64be6997cc72a6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:45:39.731585 3490722 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0315 07:45:39.742859 3490722 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.76.2
I0315 07:45:39.742895 3490722 kubeadm.go:591] duration metric: took 23.499227ms to restartPrimaryControlPlane
I0315 07:45:39.742904 3490722 kubeadm.go:393] duration metric: took 103.77824ms to StartCluster
I0315 07:45:39.742919 3490722 settings.go:142] acquiring lock: {Name:mk9341f71218475f44486dc55acbce7236fa3a16 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:45:39.742979 3490722 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18213-3295134/kubeconfig
I0315 07:45:39.743619 3490722 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/kubeconfig: {Name:mka8a8bb165c8233f51f8705aa64be6997cc72a6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:45:39.743840 3490722 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0315 07:45:39.746678 3490722 out.go:177] * Verifying Kubernetes components...
I0315 07:45:39.744145 3490722 config.go:182] Loaded profile config "old-k8s-version-591842": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0315 07:45:39.744163 3490722 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0315 07:45:39.748762 3490722 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-591842"
I0315 07:45:39.748798 3490722 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-591842"
W0315 07:45:39.748810 3490722 addons.go:243] addon storage-provisioner should already be in state true
I0315 07:45:39.748843 3490722 host.go:66] Checking if "old-k8s-version-591842" exists ...
I0315 07:45:39.749280 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:39.749420 3490722 addons.go:69] Setting dashboard=true in profile "old-k8s-version-591842"
I0315 07:45:39.749445 3490722 addons.go:234] Setting addon dashboard=true in "old-k8s-version-591842"
W0315 07:45:39.749455 3490722 addons.go:243] addon dashboard should already be in state true
I0315 07:45:39.749483 3490722 host.go:66] Checking if "old-k8s-version-591842" exists ...
I0315 07:45:39.749842 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:39.750141 3490722 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-591842"
I0315 07:45:39.750172 3490722 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-591842"
I0315 07:45:39.750424 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:39.750681 3490722 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0315 07:45:39.750942 3490722 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-591842"
I0315 07:45:39.750971 3490722 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-591842"
W0315 07:45:39.750979 3490722 addons.go:243] addon metrics-server should already be in state true
I0315 07:45:39.751004 3490722 host.go:66] Checking if "old-k8s-version-591842" exists ...
I0315 07:45:39.751413 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:39.823674 3490722 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0315 07:45:39.825627 3490722 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0315 07:45:39.827500 3490722 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0315 07:45:39.825667 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0315 07:45:39.829426 3490722 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0315 07:45:39.827582 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:39.831495 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0315 07:45:39.831527 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0315 07:45:39.831587 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:39.839352 3490722 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0315 07:45:39.841483 3490722 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0315 07:45:39.841504 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0315 07:45:39.841581 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:39.853521 3490722 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-591842"
W0315 07:45:39.853543 3490722 addons.go:243] addon default-storageclass should already be in state true
I0315 07:45:39.853570 3490722 host.go:66] Checking if "old-k8s-version-591842" exists ...
I0315 07:45:39.853963 3490722 cli_runner.go:164] Run: docker container inspect old-k8s-version-591842 --format={{.State.Status}}
I0315 07:45:39.895229 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:39.923095 3490722 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:39.923116 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0315 07:45:39.923174 3490722 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-591842
I0315 07:45:39.927223 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:39.936021 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:39.956852 3490722 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36975 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/old-k8s-version-591842/id_rsa Username:docker}
I0315 07:45:40.008007 3490722 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0315 07:45:40.056764 3490722 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-591842" to be "Ready" ...
I0315 07:45:40.071430 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0315 07:45:40.150435 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:40.190030 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0315 07:45:40.190113 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0315 07:45:40.231798 3490722 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0315 07:45:40.231873 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0315 07:45:40.275767 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0315 07:45:40.275790 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0315 07:45:40.319610 3490722 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0315 07:45:40.319635 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
W0315 07:45:40.356109 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.356145 3490722 retry.go:31] will retry after 295.704529ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.402677 3490722 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0315 07:45:40.402704 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0315 07:45:40.403009 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0315 07:45:40.403025 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0315 07:45:40.465816 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0315 07:45:40.465835 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0315 07:45:40.478364 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:40.484358 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.484433 3490722 retry.go:31] will retry after 338.188739ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.534778 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0315 07:45:40.534857 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0315 07:45:40.565132 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0315 07:45:40.565210 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0315 07:45:40.605074 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0315 07:45:40.605146 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0315 07:45:40.652407 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0315 07:45:40.694644 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0315 07:45:40.694716 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
W0315 07:45:40.750978 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.751068 3490722 retry.go:31] will retry after 306.212544ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.779952 3490722 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0315 07:45:40.780023 3490722 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
W0315 07:45:40.817048 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.817134 3490722 retry.go:31] will retry after 493.270705ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:40.823224 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:40.834462 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:41.017380 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.017456 3490722 retry.go:31] will retry after 282.135277ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0315 07:45:41.017570 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.017608 3490722 retry.go:31] will retry after 214.229924ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.059318 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0315 07:45:41.231989 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:41.242716 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.242752 3490722 retry.go:31] will retry after 370.832184ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.300032 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:41.311312 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0315 07:45:41.503700 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.503735 3490722 retry.go:31] will retry after 382.859403ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0315 07:45:41.573797 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.573832 3490722 retry.go:31] will retry after 421.507753ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0315 07:45:41.597568 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.597596 3490722 retry.go:31] will retry after 769.983218ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.613908 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:41.744066 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.744096 3490722 retry.go:31] will retry after 759.762667ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:41.886772 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0315 07:45:41.996261 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0315 07:45:42.019053 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.019156 3490722 retry.go:31] will retry after 650.163603ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.058807 3490722 node_ready.go:53] error getting node "old-k8s-version-591842": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-591842": dial tcp 192.168.76.2:8443: connect: connection refused
W0315 07:45:42.157339 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.157375 3490722 retry.go:31] will retry after 881.20092ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.367744 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0315 07:45:42.475343 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.475393 3490722 retry.go:31] will retry after 586.68407ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.504685 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:42.632293 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.632373 3490722 retry.go:31] will retry after 787.041708ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.670419 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:42.808605 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:42.808684 3490722 retry.go:31] will retry after 1.016403314s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.038796 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:43.062569 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0315 07:45:43.253111 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.253140 3490722 retry.go:31] will retry after 734.956909ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0315 07:45:43.268517 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.268546 3490722 retry.go:31] will retry after 1.198707939s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.419783 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:43.572010 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.572093 3490722 retry.go:31] will retry after 1.030643618s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.825349 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:43.914939 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.914981 3490722 retry.go:31] will retry after 1.593274041s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:43.989165 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0315 07:45:44.079597 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:44.079693 3490722 retry.go:31] will retry after 1.255103362s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:44.468126 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0315 07:45:44.544751 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:44.544783 3490722 retry.go:31] will retry after 1.986996585s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:44.558294 3490722 node_ready.go:53] error getting node "old-k8s-version-591842": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-591842": dial tcp 192.168.76.2:8443: connect: connection refused
I0315 07:45:44.603560 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:44.685563 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:44.685645 3490722 retry.go:31] will retry after 2.532501055s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:45.335058 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0315 07:45:45.427626 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:45.427686 3490722 retry.go:31] will retry after 1.730729892s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:45.508997 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:45.585805 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:45.585840 3490722 retry.go:31] will retry after 2.386594212s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:46.532002 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0315 07:45:46.558575 3490722 node_ready.go:53] error getting node "old-k8s-version-591842": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-591842": dial tcp 192.168.76.2:8443: connect: connection refused
W0315 07:45:46.612477 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:46.612504 3490722 retry.go:31] will retry after 3.523071336s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:47.158904 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:47.218334 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0315 07:45:47.230127 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:47.230168 3490722 retry.go:31] will retry after 5.073297119s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0315 07:45:47.294189 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:47.294220 3490722 retry.go:31] will retry after 3.03442375s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:47.973432 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0315 07:45:48.076816 3490722 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:48.076858 3490722 retry.go:31] will retry after 4.031308819s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0315 07:45:50.136270 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0315 07:45:50.329440 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0315 07:45:52.108772 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0315 07:45:52.304274 3490722 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0315 07:45:56.324907 3490722 node_ready.go:49] node "old-k8s-version-591842" has status "Ready":"True"
I0315 07:45:56.324939 3490722 node_ready.go:38] duration metric: took 16.267648297s for node "old-k8s-version-591842" to be "Ready" ...
I0315 07:45:56.324950 3490722 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0315 07:45:56.439554 3490722 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-5zhc9" in "kube-system" namespace to be "Ready" ...
I0315 07:45:56.580888 3490722 pod_ready.go:92] pod "coredns-74ff55c5b-5zhc9" in "kube-system" namespace has status "Ready":"True"
I0315 07:45:56.580908 3490722 pod_ready.go:81] duration metric: took 141.32728ms for pod "coredns-74ff55c5b-5zhc9" in "kube-system" namespace to be "Ready" ...
I0315 07:45:56.580920 3490722 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:45:57.445352 3490722 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (7.309045421s)
I0315 07:45:57.445455 3490722 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.115989509s)
I0315 07:45:57.445468 3490722 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-591842"
I0315 07:45:57.592288 3490722 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (5.483462834s)
I0315 07:45:57.594766 3490722 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-591842 addons enable metrics-server
I0315 07:45:57.592445 3490722 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (5.288144871s)
I0315 07:45:57.607531 3490722 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0315 07:45:57.609525 3490722 addons.go:505] duration metric: took 17.865363033s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0315 07:45:58.587630 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:00.593736 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:03.088532 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:05.092964 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:07.587661 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:09.588704 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:12.089068 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:14.588202 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:17.087714 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:19.590875 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:21.595824 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:24.089505 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:26.586843 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:28.587590 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:31.096810 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:33.586723 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:35.587751 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:37.609272 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:40.088641 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:42.089851 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:44.586855 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:46.587063 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:48.588260 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:51.091727 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:53.588199 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:56.088829 3490722 pod_ready.go:102] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:46:58.086564 3490722 pod_ready.go:92] pod "etcd-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"True"
I0315 07:46:58.086592 3490722 pod_ready.go:81] duration metric: took 1m1.505664769s for pod "etcd-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:46:58.086608 3490722 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:46:58.092323 3490722 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"True"
I0315 07:46:58.092399 3490722 pod_ready.go:81] duration metric: took 5.78233ms for pod "kube-apiserver-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:46:58.092430 3490722 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:47:00.132617 3490722 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"True"
I0315 07:47:00.132695 3490722 pod_ready.go:81] duration metric: took 2.040241761s for pod "kube-controller-manager-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:47:00.132726 3490722 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-pdn2n" in "kube-system" namespace to be "Ready" ...
I0315 07:47:00.165121 3490722 pod_ready.go:92] pod "kube-proxy-pdn2n" in "kube-system" namespace has status "Ready":"True"
I0315 07:47:00.165167 3490722 pod_ready.go:81] duration metric: took 32.412512ms for pod "kube-proxy-pdn2n" in "kube-system" namespace to be "Ready" ...
I0315 07:47:00.165183 3490722 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:47:02.172416 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:04.671598 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:06.671859 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:09.171153 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:11.172141 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:13.172616 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:15.174436 3490722 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:16.670860 3490722 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace has status "Ready":"True"
I0315 07:47:16.670887 3490722 pod_ready.go:81] duration metric: took 16.505696169s for pod "kube-scheduler-old-k8s-version-591842" in "kube-system" namespace to be "Ready" ...
I0315 07:47:16.670899 3490722 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace to be "Ready" ...
I0315 07:47:18.676148 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:20.677149 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:23.177510 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:25.178450 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:27.180486 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:29.681585 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:32.177892 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:34.677091 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:36.677208 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:39.176547 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:41.177100 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:43.678716 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:46.176844 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:48.677193 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:51.177733 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:53.178577 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:55.676892 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:47:57.677077 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:00.218163 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:02.677200 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:05.177821 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:07.676528 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:09.677363 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:12.176977 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:14.178047 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:16.676389 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:18.677490 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:21.177447 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:23.676806 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:26.176859 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:28.676490 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:30.677553 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:32.677687 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:35.177917 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:37.203184 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:39.677066 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:41.677450 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:43.680417 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:46.177075 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:48.178046 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:50.677480 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:53.176783 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:55.177798 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:57.178054 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:48:59.178669 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:01.677266 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:04.176806 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:06.176940 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:08.677227 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:11.177179 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:13.177464 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:15.178146 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:17.677076 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:20.177565 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:22.678851 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:25.178131 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:27.678624 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:30.177609 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:32.677675 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:34.678026 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:37.177468 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:39.677035 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:41.677587 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:43.677681 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:46.176790 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:48.179036 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:50.677102 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:52.677513 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:55.178196 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:57.677505 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:49:59.683771 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:02.177341 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:04.177710 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:06.676980 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:08.677377 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:11.176867 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:13.179158 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:15.676683 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:18.179596 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:20.677883 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:23.176574 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:25.177434 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:27.177893 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:29.678095 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:31.685790 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:34.176272 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:36.178511 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:38.676751 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:40.678288 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:43.177996 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:45.186438 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:47.678227 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:50.177996 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:52.677058 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:55.177803 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:50:57.676940 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:00.192744 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:02.678433 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:05.177402 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:07.686339 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:10.178061 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:12.677423 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:14.677510 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:16.677351 3490722 pod_ready.go:81] duration metric: took 4m0.006438222s for pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace to be "Ready" ...
E0315 07:51:16.677377 3490722 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0315 07:51:16.677387 3490722 pod_ready.go:38] duration metric: took 5m20.352426308s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0315 07:51:16.677401 3490722 api_server.go:52] waiting for apiserver process to appear ...
I0315 07:51:16.677430 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0315 07:51:16.677492 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0315 07:51:16.741347 3490722 cri.go:89] found id: "3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:16.741365 3490722 cri.go:89] found id: "aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:16.741370 3490722 cri.go:89] found id: ""
I0315 07:51:16.741377 3490722 logs.go:276] 2 containers: [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5]
I0315 07:51:16.741429 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.745174 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.748785 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0315 07:51:16.748846 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0315 07:51:16.809837 3490722 cri.go:89] found id: "3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:16.809901 3490722 cri.go:89] found id: "288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:16.809917 3490722 cri.go:89] found id: ""
I0315 07:51:16.809925 3490722 logs.go:276] 2 containers: [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf]
I0315 07:51:16.809988 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.821230 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.832434 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0315 07:51:16.832513 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0315 07:51:16.893799 3490722 cri.go:89] found id: "9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:16.893823 3490722 cri.go:89] found id: "7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:16.893828 3490722 cri.go:89] found id: ""
I0315 07:51:16.893836 3490722 logs.go:276] 2 containers: [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc]
I0315 07:51:16.893889 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.898028 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.902368 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0315 07:51:16.902437 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0315 07:51:16.958613 3490722 cri.go:89] found id: "19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:16.958636 3490722 cri.go:89] found id: "eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:16.958642 3490722 cri.go:89] found id: ""
I0315 07:51:16.958649 3490722 logs.go:276] 2 containers: [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d]
I0315 07:51:16.958703 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.962711 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.966605 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0315 07:51:16.966681 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0315 07:51:17.016946 3490722 cri.go:89] found id: "24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:17.016970 3490722 cri.go:89] found id: "1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:17.016976 3490722 cri.go:89] found id: ""
I0315 07:51:17.016985 3490722 logs.go:276] 2 containers: [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439]
I0315 07:51:17.017041 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.020761 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.027026 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0315 07:51:17.027194 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0315 07:51:17.081451 3490722 cri.go:89] found id: "39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:17.081475 3490722 cri.go:89] found id: "6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:17.081480 3490722 cri.go:89] found id: ""
I0315 07:51:17.081487 3490722 logs.go:276] 2 containers: [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d]
I0315 07:51:17.081544 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.085402 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.088913 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0315 07:51:17.088999 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0315 07:51:17.152035 3490722 cri.go:89] found id: "d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:17.152107 3490722 cri.go:89] found id: "7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:17.152125 3490722 cri.go:89] found id: ""
I0315 07:51:17.152146 3490722 logs.go:276] 2 containers: [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e]
I0315 07:51:17.152229 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.156132 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.159619 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0315 07:51:17.159727 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0315 07:51:17.208743 3490722 cri.go:89] found id: "141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:17.208812 3490722 cri.go:89] found id: ""
I0315 07:51:17.208834 3490722 logs.go:276] 1 containers: [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c]
I0315 07:51:17.208916 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.212643 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0315 07:51:17.212756 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0315 07:51:17.276269 3490722 cri.go:89] found id: "ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:17.276352 3490722 cri.go:89] found id: "4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:17.276372 3490722 cri.go:89] found id: ""
I0315 07:51:17.276394 3490722 logs.go:276] 2 containers: [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2]
I0315 07:51:17.276485 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.281157 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.285880 3490722 logs.go:123] Gathering logs for kube-apiserver [aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5] ...
I0315 07:51:17.285975 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:17.383966 3490722 logs.go:123] Gathering logs for etcd [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe] ...
I0315 07:51:17.383996 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:17.452945 3490722 logs.go:123] Gathering logs for etcd [288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf] ...
I0315 07:51:17.453085 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:17.518825 3490722 logs.go:123] Gathering logs for kube-proxy [1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439] ...
I0315 07:51:17.518972 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:17.582975 3490722 logs.go:123] Gathering logs for kube-controller-manager [6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d] ...
I0315 07:51:17.582999 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:17.721400 3490722 logs.go:123] Gathering logs for kindnet [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842] ...
I0315 07:51:17.721438 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:17.782289 3490722 logs.go:123] Gathering logs for kubelet ...
I0315 07:51:17.782317 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0315 07:51:17.844998 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151553 663 reflector.go:138] object-"kube-system"/"metrics-server-token-h2gnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-h2gnq" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845272 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151674 663 reflector.go:138] object-"default"/"default-token-82cb9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-82cb9" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845526 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151731 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845762 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151795 663 reflector.go:138] object-"kube-system"/"coredns-token-vw5pl": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-vw5pl" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845999 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151873 663 reflector.go:138] object-"kube-system"/"kindnet-token-jrrqr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jrrqr" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846238 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151952 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-vqzhx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vqzhx" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846467 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152014 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846715 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152079 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-hnqzn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-hnqzn" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.857570 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:58 old-k8s-version-591842 kubelet[663]: E0315 07:45:58.854597 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.857857 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:59 old-k8s-version-591842 kubelet[663]: E0315 07:45:59.802434 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.861219 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:14 old-k8s-version-591842 kubelet[663]: E0315 07:46:14.653468 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.863383 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:20 old-k8s-version-591842 kubelet[663]: E0315 07:46:20.888372 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.863745 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:21 old-k8s-version-591842 kubelet[663]: E0315 07:46:21.893270 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.864106 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:22 old-k8s-version-591842 kubelet[663]: E0315 07:46:22.992818 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.864314 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:25 old-k8s-version-591842 kubelet[663]: E0315 07:46:25.647507 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.865123 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:30 old-k8s-version-591842 kubelet[663]: E0315 07:46:30.928822 663 pod_workers.go:191] Error syncing pod 1b02bdb3-5934-4002-980c-769d1de68357 ("storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"
W0315 07:51:17.865704 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:35 old-k8s-version-591842 kubelet[663]: E0315 07:46:35.946453 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.868835 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:37 old-k8s-version-591842 kubelet[663]: E0315 07:46:37.659626 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.869661 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:42 old-k8s-version-591842 kubelet[663]: E0315 07:46:42.993141 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.869869 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:48 old-k8s-version-591842 kubelet[663]: E0315 07:46:48.651057 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.870811 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:58 old-k8s-version-591842 kubelet[663]: E0315 07:46:58.015895 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.871197 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:02 old-k8s-version-591842 kubelet[663]: E0315 07:47:02.992859 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.871475 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:03 old-k8s-version-591842 kubelet[663]: E0315 07:47:03.649062 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.871720 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:15 old-k8s-version-591842 kubelet[663]: E0315 07:47:15.644773 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.872103 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:16 old-k8s-version-591842 kubelet[663]: E0315 07:47:16.645024 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.874664 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:29 old-k8s-version-591842 kubelet[663]: E0315 07:47:29.656112 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.875037 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:30 old-k8s-version-591842 kubelet[663]: E0315 07:47:30.644347 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.875270 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:42 old-k8s-version-591842 kubelet[663]: E0315 07:47:42.645173 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.875894 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:45 old-k8s-version-591842 kubelet[663]: E0315 07:47:45.161682 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.876256 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:52 old-k8s-version-591842 kubelet[663]: E0315 07:47:52.992815 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.876463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:54 old-k8s-version-591842 kubelet[663]: E0315 07:47:54.645803 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.876813 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.644489 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.877020 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.645236 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.877369 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.644500 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.877577 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.645259 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.877786 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:29 old-k8s-version-591842 kubelet[663]: E0315 07:48:29.644702 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.878133 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:33 old-k8s-version-591842 kubelet[663]: E0315 07:48:33.645925 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.878366 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:42 old-k8s-version-591842 kubelet[663]: E0315 07:48:42.644688 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.878717 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:45 old-k8s-version-591842 kubelet[663]: E0315 07:48:45.644849 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.881377 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:57 old-k8s-version-591842 kubelet[663]: E0315 07:48:57.654804 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.881741 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:59 old-k8s-version-591842 kubelet[663]: E0315 07:48:59.644341 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.881947 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:09 old-k8s-version-591842 kubelet[663]: E0315 07:49:09.653348 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.882559 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:14 old-k8s-version-591842 kubelet[663]: E0315 07:49:14.388467 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.882911 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:22 old-k8s-version-591842 kubelet[663]: E0315 07:49:22.993103 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.883162 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:23 old-k8s-version-591842 kubelet[663]: E0315 07:49:23.645937 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.883370 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:35 old-k8s-version-591842 kubelet[663]: E0315 07:49:35.645912 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.883715 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:36 old-k8s-version-591842 kubelet[663]: E0315 07:49:36.644319 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.883929 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:47 old-k8s-version-591842 kubelet[663]: E0315 07:49:47.658879 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.884276 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:50 old-k8s-version-591842 kubelet[663]: E0315 07:49:50.644297 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.884485 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:02 old-k8s-version-591842 kubelet[663]: E0315 07:50:02.644922 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.884840 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:05 old-k8s-version-591842 kubelet[663]: E0315 07:50:05.644906 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.885212 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.645285 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.885484 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.647267 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.889275 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: E0315 07:50:28.644332 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.889531 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:29 old-k8s-version-591842 kubelet[663]: E0315 07:50:29.645326 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.889739 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:42 old-k8s-version-591842 kubelet[663]: E0315 07:50:42.644973 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.890100 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.890309 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.890655 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.891006 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.891245 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0315 07:51:17.891273 3490722 logs.go:123] Gathering logs for dmesg ...
I0315 07:51:17.891305 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0315 07:51:17.929063 3490722 logs.go:123] Gathering logs for kindnet [7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e] ...
I0315 07:51:17.929137 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:17.993303 3490722 logs.go:123] Gathering logs for kube-scheduler [eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d] ...
I0315 07:51:17.993327 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:18.145483 3490722 logs.go:123] Gathering logs for container status ...
I0315 07:51:18.145518 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0315 07:51:18.232250 3490722 logs.go:123] Gathering logs for kube-apiserver [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465] ...
I0315 07:51:18.232279 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:18.307131 3490722 logs.go:123] Gathering logs for coredns [7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc] ...
I0315 07:51:18.307162 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:18.372534 3490722 logs.go:123] Gathering logs for kubernetes-dashboard [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c] ...
I0315 07:51:18.372563 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:18.423802 3490722 logs.go:123] Gathering logs for coredns [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e] ...
I0315 07:51:18.423835 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:18.484538 3490722 logs.go:123] Gathering logs for kube-controller-manager [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2] ...
I0315 07:51:18.484567 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:18.734982 3490722 logs.go:123] Gathering logs for kube-proxy [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3] ...
I0315 07:51:18.735022 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:18.989000 3490722 logs.go:123] Gathering logs for storage-provisioner [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803] ...
I0315 07:51:18.989036 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:19.097844 3490722 logs.go:123] Gathering logs for storage-provisioner [4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2] ...
I0315 07:51:19.097876 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:19.152808 3490722 logs.go:123] Gathering logs for containerd ...
I0315 07:51:19.152845 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0315 07:51:19.226776 3490722 logs.go:123] Gathering logs for describe nodes ...
I0315 07:51:19.226813 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0315 07:51:19.465101 3490722 logs.go:123] Gathering logs for kube-scheduler [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423] ...
I0315 07:51:19.465178 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:19.520188 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:19.520213 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0315 07:51:19.520268 3490722 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0315 07:51:19.520277 3490722 out.go:239] Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520285 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:19.520293 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520302 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520311 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0315 07:51:19.520322 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:19.520329 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:29.521502 3490722 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0315 07:51:29.538525 3490722 api_server.go:72] duration metric: took 5m49.79464477s to wait for apiserver process to appear ...
I0315 07:51:29.538551 3490722 api_server.go:88] waiting for apiserver healthz status ...
I0315 07:51:29.538596 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0315 07:51:29.538676 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0315 07:51:29.599859 3490722 cri.go:89] found id: "3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:29.599884 3490722 cri.go:89] found id: "aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:29.599889 3490722 cri.go:89] found id: ""
I0315 07:51:29.599897 3490722 logs.go:276] 2 containers: [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5]
I0315 07:51:29.599971 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.605961 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.610920 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0315 07:51:29.611054 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0315 07:51:29.685499 3490722 cri.go:89] found id: "3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:29.685526 3490722 cri.go:89] found id: "288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:29.685530 3490722 cri.go:89] found id: ""
I0315 07:51:29.685537 3490722 logs.go:276] 2 containers: [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf]
I0315 07:51:29.685610 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.690445 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.695366 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0315 07:51:29.695452 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0315 07:51:29.771248 3490722 cri.go:89] found id: "9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:29.771267 3490722 cri.go:89] found id: "7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:29.771272 3490722 cri.go:89] found id: ""
I0315 07:51:29.771279 3490722 logs.go:276] 2 containers: [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc]
I0315 07:51:29.771337 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.776267 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.793698 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0315 07:51:29.793769 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0315 07:51:29.845789 3490722 cri.go:89] found id: "19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:29.845851 3490722 cri.go:89] found id: "eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:29.845878 3490722 cri.go:89] found id: ""
I0315 07:51:29.845898 3490722 logs.go:276] 2 containers: [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d]
I0315 07:51:29.845985 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.851019 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.855662 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0315 07:51:29.855846 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0315 07:51:29.914978 3490722 cri.go:89] found id: "24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:29.915050 3490722 cri.go:89] found id: "1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:29.915095 3490722 cri.go:89] found id: ""
I0315 07:51:29.915121 3490722 logs.go:276] 2 containers: [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439]
I0315 07:51:29.915205 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.923722 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.927822 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0315 07:51:29.927942 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0315 07:51:29.989735 3490722 cri.go:89] found id: "39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:29.989797 3490722 cri.go:89] found id: "6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:29.989825 3490722 cri.go:89] found id: ""
I0315 07:51:29.989846 3490722 logs.go:276] 2 containers: [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d]
I0315 07:51:29.989933 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.994641 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.998628 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0315 07:51:29.998750 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0315 07:51:30.121952 3490722 cri.go:89] found id: "d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:30.122032 3490722 cri.go:89] found id: "7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:30.122054 3490722 cri.go:89] found id: ""
I0315 07:51:30.122083 3490722 logs.go:276] 2 containers: [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e]
I0315 07:51:30.122196 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.127774 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.144595 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0315 07:51:30.144766 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0315 07:51:30.219866 3490722 cri.go:89] found id: "141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:30.219939 3490722 cri.go:89] found id: ""
I0315 07:51:30.219962 3490722 logs.go:276] 1 containers: [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c]
I0315 07:51:30.220076 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.226314 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0315 07:51:30.226500 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0315 07:51:30.285822 3490722 cri.go:89] found id: "ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:30.285848 3490722 cri.go:89] found id: "4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:30.285854 3490722 cri.go:89] found id: ""
I0315 07:51:30.285862 3490722 logs.go:276] 2 containers: [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2]
I0315 07:51:30.285924 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.291346 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.296522 3490722 logs.go:123] Gathering logs for kubernetes-dashboard [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c] ...
I0315 07:51:30.296594 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:30.367022 3490722 logs.go:123] Gathering logs for storage-provisioner [4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2] ...
I0315 07:51:30.367147 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:30.420538 3490722 logs.go:123] Gathering logs for containerd ...
I0315 07:51:30.420567 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0315 07:51:30.496522 3490722 logs.go:123] Gathering logs for container status ...
I0315 07:51:30.496650 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0315 07:51:30.579010 3490722 logs.go:123] Gathering logs for kube-scheduler [eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d] ...
I0315 07:51:30.579049 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:30.630592 3490722 logs.go:123] Gathering logs for kube-proxy [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3] ...
I0315 07:51:30.630626 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:30.678109 3490722 logs.go:123] Gathering logs for kindnet [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842] ...
I0315 07:51:30.678138 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:30.723399 3490722 logs.go:123] Gathering logs for kube-apiserver [aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5] ...
I0315 07:51:30.723428 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:30.797968 3490722 logs.go:123] Gathering logs for dmesg ...
I0315 07:51:30.798005 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0315 07:51:30.816622 3490722 logs.go:123] Gathering logs for describe nodes ...
I0315 07:51:30.816653 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0315 07:51:30.972164 3490722 logs.go:123] Gathering logs for kube-apiserver [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465] ...
I0315 07:51:30.972197 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:31.050833 3490722 logs.go:123] Gathering logs for kindnet [7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e] ...
I0315 07:51:31.050875 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:31.123735 3490722 logs.go:123] Gathering logs for storage-provisioner [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803] ...
I0315 07:51:31.123775 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:31.190207 3490722 logs.go:123] Gathering logs for coredns [7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc] ...
I0315 07:51:31.190236 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:31.249826 3490722 logs.go:123] Gathering logs for kube-controller-manager [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2] ...
I0315 07:51:31.249855 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:31.354117 3490722 logs.go:123] Gathering logs for kube-controller-manager [6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d] ...
I0315 07:51:31.354155 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:31.441541 3490722 logs.go:123] Gathering logs for coredns [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e] ...
I0315 07:51:31.441581 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:31.515474 3490722 logs.go:123] Gathering logs for kube-scheduler [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423] ...
I0315 07:51:31.515503 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:31.589337 3490722 logs.go:123] Gathering logs for kube-proxy [1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439] ...
I0315 07:51:31.589367 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:31.679207 3490722 logs.go:123] Gathering logs for kubelet ...
I0315 07:51:31.679237 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0315 07:51:31.765777 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151553 663 reflector.go:138] object-"kube-system"/"metrics-server-token-h2gnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-h2gnq" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766009 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151674 663 reflector.go:138] object-"default"/"default-token-82cb9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-82cb9" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766219 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151731 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766444 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151795 663 reflector.go:138] object-"kube-system"/"coredns-token-vw5pl": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-vw5pl" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766666 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151873 663 reflector.go:138] object-"kube-system"/"kindnet-token-jrrqr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jrrqr" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766881 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151952 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-vqzhx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vqzhx" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.767126 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152014 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.767352 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152079 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-hnqzn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-hnqzn" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.777873 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:58 old-k8s-version-591842 kubelet[663]: E0315 07:45:58.854597 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.778081 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:59 old-k8s-version-591842 kubelet[663]: E0315 07:45:59.802434 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.784967 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:14 old-k8s-version-591842 kubelet[663]: E0315 07:46:14.653468 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.787050 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:20 old-k8s-version-591842 kubelet[663]: E0315 07:46:20.888372 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.788741 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:21 old-k8s-version-591842 kubelet[663]: E0315 07:46:21.893270 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.789087 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:22 old-k8s-version-591842 kubelet[663]: E0315 07:46:22.992818 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.789271 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:25 old-k8s-version-591842 kubelet[663]: E0315 07:46:25.647507 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.790036 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:30 old-k8s-version-591842 kubelet[663]: E0315 07:46:30.928822 663 pod_workers.go:191] Error syncing pod 1b02bdb3-5934-4002-980c-769d1de68357 ("storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"
W0315 07:51:31.790614 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:35 old-k8s-version-591842 kubelet[663]: E0315 07:46:35.946453 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.793030 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:37 old-k8s-version-591842 kubelet[663]: E0315 07:46:37.659626 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.793817 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:42 old-k8s-version-591842 kubelet[663]: E0315 07:46:42.993141 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.794004 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:48 old-k8s-version-591842 kubelet[663]: E0315 07:46:48.651057 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.794583 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:58 old-k8s-version-591842 kubelet[663]: E0315 07:46:58.015895 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.794910 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:02 old-k8s-version-591842 kubelet[663]: E0315 07:47:02.992859 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.797506 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:03 old-k8s-version-591842 kubelet[663]: E0315 07:47:03.649062 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.800132 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:15 old-k8s-version-591842 kubelet[663]: E0315 07:47:15.644773 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.800463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:16 old-k8s-version-591842 kubelet[663]: E0315 07:47:16.645024 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.802857 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:29 old-k8s-version-591842 kubelet[663]: E0315 07:47:29.656112 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.803187 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:30 old-k8s-version-591842 kubelet[663]: E0315 07:47:30.644347 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.803376 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:42 old-k8s-version-591842 kubelet[663]: E0315 07:47:42.645173 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.803959 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:45 old-k8s-version-591842 kubelet[663]: E0315 07:47:45.161682 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804281 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:52 old-k8s-version-591842 kubelet[663]: E0315 07:47:52.992815 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:54 old-k8s-version-591842 kubelet[663]: E0315 07:47:54.645803 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.804784 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.644489 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804966 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.645236 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805298 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.644500 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.805483 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.645259 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805664 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:29 old-k8s-version-591842 kubelet[663]: E0315 07:48:29.644702 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805987 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:33 old-k8s-version-591842 kubelet[663]: E0315 07:48:33.645925 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.806168 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:42 old-k8s-version-591842 kubelet[663]: E0315 07:48:42.644688 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.806493 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:45 old-k8s-version-591842 kubelet[663]: E0315 07:48:45.644849 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.808911 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:57 old-k8s-version-591842 kubelet[663]: E0315 07:48:57.654804 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.809237 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:59 old-k8s-version-591842 kubelet[663]: E0315 07:48:59.644341 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.809453 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:09 old-k8s-version-591842 kubelet[663]: E0315 07:49:09.653348 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.810032 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:14 old-k8s-version-591842 kubelet[663]: E0315 07:49:14.388467 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.810354 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:22 old-k8s-version-591842 kubelet[663]: E0315 07:49:22.993103 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.810536 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:23 old-k8s-version-591842 kubelet[663]: E0315 07:49:23.645937 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.810717 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:35 old-k8s-version-591842 kubelet[663]: E0315 07:49:35.645912 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.811038 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:36 old-k8s-version-591842 kubelet[663]: E0315 07:49:36.644319 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.813162 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:47 old-k8s-version-591842 kubelet[663]: E0315 07:49:47.658879 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.813501 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:50 old-k8s-version-591842 kubelet[663]: E0315 07:49:50.644297 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.813686 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:02 old-k8s-version-591842 kubelet[663]: E0315 07:50:02.644922 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.814013 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:05 old-k8s-version-591842 kubelet[663]: E0315 07:50:05.644906 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.814335 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.645285 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.814518 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.647267 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.814841 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: E0315 07:50:28.644332 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.815023 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:29 old-k8s-version-591842 kubelet[663]: E0315 07:50:29.645326 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816134 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:42 old-k8s-version-591842 kubelet[663]: E0315 07:50:42.644973 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816475 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.816660 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816996 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.817320 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.817501 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.817684 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.818007 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
I0315 07:51:31.818018 3490722 logs.go:123] Gathering logs for etcd [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe] ...
I0315 07:51:31.818032 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:31.903966 3490722 logs.go:123] Gathering logs for etcd [288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf] ...
I0315 07:51:31.903997 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:31.986316 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:31.986344 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0315 07:51:31.986390 3490722 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0315 07:51:31.986404 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.986414 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.986428 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.986435 3490722 out.go:239] Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.986449 3490722 out.go:239] Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
I0315 07:51:31.986457 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:31.986464 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:41.987133 3490722 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0315 07:51:41.998532 3490722 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0315 07:51:42.008114 3490722 out.go:177]
W0315 07:51:42.013913 3490722 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0315 07:51:42.013959 3490722 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0315 07:51:42.013981 3490722 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0315 07:51:42.013988 3490722 out.go:239] *
*
W0315 07:51:42.015597 3490722 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0315 07:51:42.018503 3490722 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-591842 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-591842
helpers_test.go:235: (dbg) docker inspect old-k8s-version-591842:
-- stdout --
[
{
"Id": "18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc",
"Created": "2024-03-15T07:42:21.029029564Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 3490914,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-03-15T07:45:32.360092308Z",
"FinishedAt": "2024-03-15T07:45:31.249186001Z"
},
"Image": "sha256:db62270b4bb0cfcde696782f7a6322baca275275e31814ce9fd8998407bf461e",
"ResolvConfPath": "/var/lib/docker/containers/18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc/hostname",
"HostsPath": "/var/lib/docker/containers/18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc/hosts",
"LogPath": "/var/lib/docker/containers/18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc/18fa161c9b305c812c8cb0139978d2076a2cbfd7c3b54015de2a346b49e84efc-json.log",
"Name": "/old-k8s-version-591842",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-591842:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-591842",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/5cf4b5005a795b67c812375de157beba0f565cab11117ac227fe78989c7c1632-init/diff:/var/lib/docker/overlay2/81bfb75b66991fc99a81a39de84c7e82ece5b807050cd14d22a1050d39339cc7/diff",
"MergedDir": "/var/lib/docker/overlay2/5cf4b5005a795b67c812375de157beba0f565cab11117ac227fe78989c7c1632/merged",
"UpperDir": "/var/lib/docker/overlay2/5cf4b5005a795b67c812375de157beba0f565cab11117ac227fe78989c7c1632/diff",
"WorkDir": "/var/lib/docker/overlay2/5cf4b5005a795b67c812375de157beba0f565cab11117ac227fe78989c7c1632/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-591842",
"Source": "/var/lib/docker/volumes/old-k8s-version-591842/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-591842",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-591842",
"name.minikube.sigs.k8s.io": "old-k8s-version-591842",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "6a9f8dcd7360caf540d14a17c7fe00e3cd7f8668d7fcccc210f81fe4aa09765d",
"SandboxKey": "/var/run/docker/netns/6a9f8dcd7360",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36975"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36974"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36971"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36973"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36972"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-591842": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"18fa161c9b30",
"old-k8s-version-591842"
],
"MacAddress": "02:42:c0:a8:4c:02",
"NetworkID": "3a5282fbcdd526425fdd031a05b57d603c6c505a01ddafd1bc89f63881ca5da8",
"EndpointID": "508a6c0bafdc514c09d7a945d113f5288672f9980150a624c872249c913dfab3",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-591842",
"18fa161c9b30"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-591842 -n old-k8s-version-591842
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-591842 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-591842 logs -n 25: (2.069631704s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
| start | -p cert-expiration-764294 | cert-expiration-764294 | jenkins | v1.32.0 | 15 Mar 24 07:41 UTC | 15 Mar 24 07:41 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-770281 | force-systemd-env-770281 | jenkins | v1.32.0 | 15 Mar 24 07:41 UTC | 15 Mar 24 07:41 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-770281 | force-systemd-env-770281 | jenkins | v1.32.0 | 15 Mar 24 07:41 UTC | 15 Mar 24 07:41 UTC |
| start | -p cert-options-342304 | cert-options-342304 | jenkins | v1.32.0 | 15 Mar 24 07:41 UTC | 15 Mar 24 07:42 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-342304 ssh | cert-options-342304 | jenkins | v1.32.0 | 15 Mar 24 07:42 UTC | 15 Mar 24 07:42 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-342304 -- sudo | cert-options-342304 | jenkins | v1.32.0 | 15 Mar 24 07:42 UTC | 15 Mar 24 07:42 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-342304 | cert-options-342304 | jenkins | v1.32.0 | 15 Mar 24 07:42 UTC | 15 Mar 24 07:42 UTC |
| start | -p old-k8s-version-591842 | old-k8s-version-591842 | jenkins | v1.32.0 | 15 Mar 24 07:42 UTC | 15 Mar 24 07:45 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-764294 | cert-expiration-764294 | jenkins | v1.32.0 | 15 Mar 24 07:44 UTC | 15 Mar 24 07:44 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-764294 | cert-expiration-764294 | jenkins | v1.32.0 | 15 Mar 24 07:44 UTC | 15 Mar 24 07:44 UTC |
| start | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:44 UTC | 15 Mar 24 07:45 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --apiserver-port=8444 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.28.4 | | | | | |
| addons | enable metrics-server -p old-k8s-version-591842 | old-k8s-version-591842 | jenkins | v1.32.0 | 15 Mar 24 07:45 UTC | 15 Mar 24 07:45 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-591842 | old-k8s-version-591842 | jenkins | v1.32.0 | 15 Mar 24 07:45 UTC | 15 Mar 24 07:45 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-591842 | old-k8s-version-591842 | jenkins | v1.32.0 | 15 Mar 24 07:45 UTC | 15 Mar 24 07:45 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-591842 | old-k8s-version-591842 | jenkins | v1.32.0 | 15 Mar 24 07:45 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p default-k8s-diff-port-484299 | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:46 UTC | 15 Mar 24 07:46 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:46 UTC | 15 Mar 24 07:46 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p default-k8s-diff-port-484299 | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:46 UTC | 15 Mar 24 07:46 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:46 UTC | 15 Mar 24 07:50 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --apiserver-port=8444 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.28.4 | | | | | |
| image | default-k8s-diff-port-484299 | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | 15 Mar 24 07:51 UTC |
| | image list --format=json | | | | | |
| pause | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | 15 Mar 24 07:51 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | 15 Mar 24 07:51 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | 15 Mar 24 07:51 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| delete | -p | default-k8s-diff-port-484299 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | 15 Mar 24 07:51 UTC |
| | default-k8s-diff-port-484299 | | | | | |
| start | -p embed-certs-722347 | embed-certs-722347 | jenkins | v1.32.0 | 15 Mar 24 07:51 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.28.4 | | | | | |
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/03/15 07:51:06
Running on machine: ip-172-31-21-244
Binary: Built with gc go1.22.1 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0315 07:51:06.964246 3500589 out.go:291] Setting OutFile to fd 1 ...
I0315 07:51:06.966359 3500589 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:06.966374 3500589 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:06.966381 3500589 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:06.966809 3500589 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18213-3295134/.minikube/bin
I0315 07:51:06.967706 3500589 out.go:298] Setting JSON to false
I0315 07:51:06.969365 3500589 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":59611,"bootTime":1710429456,"procs":227,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I0315 07:51:06.969467 3500589 start.go:139] virtualization:
I0315 07:51:06.972760 3500589 out.go:177] * [embed-certs-722347] minikube v1.32.0 on Ubuntu 20.04 (arm64)
I0315 07:51:06.975838 3500589 out.go:177] - MINIKUBE_LOCATION=18213
I0315 07:51:06.977590 3500589 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0315 07:51:06.976000 3500589 notify.go:220] Checking for updates...
I0315 07:51:06.979643 3500589 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18213-3295134/kubeconfig
I0315 07:51:06.981630 3500589 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18213-3295134/.minikube
I0315 07:51:06.983794 3500589 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0315 07:51:06.985642 3500589 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0315 07:51:06.988451 3500589 config.go:182] Loaded profile config "old-k8s-version-591842": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0315 07:51:06.988549 3500589 driver.go:392] Setting default libvirt URI to qemu:///system
I0315 07:51:07.018396 3500589 docker.go:122] docker version: linux-25.0.4:Docker Engine - Community
I0315 07:51:07.018519 3500589 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0315 07:51:07.096934 3500589 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-15 07:51:07.086932252 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0315 07:51:07.097046 3500589 docker.go:295] overlay module found
I0315 07:51:07.099243 3500589 out.go:177] * Using the docker driver based on user configuration
I0315 07:51:07.101230 3500589 start.go:297] selected driver: docker
I0315 07:51:07.101251 3500589 start.go:901] validating driver "docker" against <nil>
I0315 07:51:07.101267 3500589 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0315 07:51:07.101926 3500589 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0315 07:51:07.156304 3500589 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-15 07:51:07.146478273 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:25.0.4 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.7]] Warnings:<nil>}}
I0315 07:51:07.156458 3500589 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0315 07:51:07.156691 3500589 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0315 07:51:07.158654 3500589 out.go:177] * Using Docker driver with root privileges
I0315 07:51:07.160820 3500589 cni.go:84] Creating CNI manager for ""
I0315 07:51:07.160843 3500589 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0315 07:51:07.160854 3500589 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0315 07:51:07.160933 3500589 start.go:340] cluster config:
{Name:embed-certs-722347 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-722347 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stati
cIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0315 07:51:07.163064 3500589 out.go:177] * Starting "embed-certs-722347" primary control-plane node in "embed-certs-722347" cluster
I0315 07:51:07.165285 3500589 cache.go:121] Beginning downloading kic base image for docker with containerd
I0315 07:51:07.168261 3500589 out.go:177] * Pulling base image v0.0.42-1710284843-18375 ...
I0315 07:51:07.170411 3500589 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0315 07:51:07.170466 3500589 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4
I0315 07:51:07.170472 3500589 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f in local docker daemon
I0315 07:51:07.170479 3500589 cache.go:56] Caching tarball of preloaded images
I0315 07:51:07.170703 3500589 preload.go:173] Found /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0315 07:51:07.170717 3500589 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on containerd
I0315 07:51:07.170815 3500589 profile.go:142] Saving config to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/config.json ...
I0315 07:51:07.170844 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/config.json: {Name:mk8e63ea88fd916c4c3f98fbf1530b6329625bc3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:07.188577 3500589 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f in local docker daemon, skipping pull
I0315 07:51:07.188600 3500589 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f exists in daemon, skipping load
I0315 07:51:07.188624 3500589 cache.go:194] Successfully downloaded all kic artifacts
I0315 07:51:07.188652 3500589 start.go:360] acquireMachinesLock for embed-certs-722347: {Name:mkbbec9ff3261837b604deb6de7373b706f2025b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0315 07:51:07.189255 3500589 start.go:364] duration metric: took 582.854µs to acquireMachinesLock for "embed-certs-722347"
I0315 07:51:07.189307 3500589 start.go:93] Provisioning new machine with config: &{Name:embed-certs-722347 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-722347 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableM
etrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0315 07:51:07.189397 3500589 start.go:125] createHost starting for "" (driver="docker")
I0315 07:51:07.686339 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:10.178061 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:07.193157 3500589 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0315 07:51:07.193404 3500589 start.go:159] libmachine.API.Create for "embed-certs-722347" (driver="docker")
I0315 07:51:07.193439 3500589 client.go:168] LocalClient.Create starting
I0315 07:51:07.193510 3500589 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem
I0315 07:51:07.193547 3500589 main.go:141] libmachine: Decoding PEM data...
I0315 07:51:07.193564 3500589 main.go:141] libmachine: Parsing certificate...
I0315 07:51:07.193637 3500589 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem
I0315 07:51:07.193661 3500589 main.go:141] libmachine: Decoding PEM data...
I0315 07:51:07.193678 3500589 main.go:141] libmachine: Parsing certificate...
I0315 07:51:07.194032 3500589 cli_runner.go:164] Run: docker network inspect embed-certs-722347 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0315 07:51:07.208543 3500589 cli_runner.go:211] docker network inspect embed-certs-722347 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0315 07:51:07.208630 3500589 network_create.go:281] running [docker network inspect embed-certs-722347] to gather additional debugging logs...
I0315 07:51:07.208649 3500589 cli_runner.go:164] Run: docker network inspect embed-certs-722347
W0315 07:51:07.224176 3500589 cli_runner.go:211] docker network inspect embed-certs-722347 returned with exit code 1
I0315 07:51:07.224208 3500589 network_create.go:284] error running [docker network inspect embed-certs-722347]: docker network inspect embed-certs-722347: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-722347 not found
I0315 07:51:07.224220 3500589 network_create.go:286] output of [docker network inspect embed-certs-722347]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-722347 not found
** /stderr **
I0315 07:51:07.224319 3500589 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0315 07:51:07.239549 3500589 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-2a726f180238 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:7e:c5:12:0f} reservation:<nil>}
I0315 07:51:07.240045 3500589 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-b9c6f1496deb IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:7b:41:1e:22} reservation:<nil>}
I0315 07:51:07.240449 3500589 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-1634a6e12e83 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:43:0d:36:ef} reservation:<nil>}
I0315 07:51:07.240784 3500589 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-3a5282fbcdd5 IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:02:42:8a:49:e4:6c} reservation:<nil>}
I0315 07:51:07.241283 3500589 network.go:206] using free private subnet 192.168.85.0/24: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4002624f80}
I0315 07:51:07.241330 3500589 network_create.go:124] attempt to create docker network embed-certs-722347 192.168.85.0/24 with gateway 192.168.85.1 and MTU of 1500 ...
I0315 07:51:07.241386 3500589 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.85.0/24 --gateway=192.168.85.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-722347 embed-certs-722347
I0315 07:51:07.316575 3500589 network_create.go:108] docker network embed-certs-722347 192.168.85.0/24 created
I0315 07:51:07.316607 3500589 kic.go:121] calculated static IP "192.168.85.2" for the "embed-certs-722347" container
I0315 07:51:07.316675 3500589 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0315 07:51:07.332547 3500589 cli_runner.go:164] Run: docker volume create embed-certs-722347 --label name.minikube.sigs.k8s.io=embed-certs-722347 --label created_by.minikube.sigs.k8s.io=true
I0315 07:51:07.348410 3500589 oci.go:103] Successfully created a docker volume embed-certs-722347
I0315 07:51:07.348490 3500589 cli_runner.go:164] Run: docker run --rm --name embed-certs-722347-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-722347 --entrypoint /usr/bin/test -v embed-certs-722347:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f -d /var/lib
I0315 07:51:08.025178 3500589 oci.go:107] Successfully prepared a docker volume embed-certs-722347
I0315 07:51:08.025221 3500589 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0315 07:51:08.025241 3500589 kic.go:194] Starting extracting preloaded images to volume ...
I0315 07:51:08.025331 3500589 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-722347:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f -I lz4 -xf /preloaded.tar -C /extractDir
I0315 07:51:12.677423 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:14.677510 3490722 pod_ready.go:102] pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace has status "Ready":"False"
I0315 07:51:16.677351 3490722 pod_ready.go:81] duration metric: took 4m0.006438222s for pod "metrics-server-9975d5f86-9j72g" in "kube-system" namespace to be "Ready" ...
E0315 07:51:16.677377 3490722 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0315 07:51:16.677387 3490722 pod_ready.go:38] duration metric: took 5m20.352426308s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0315 07:51:16.677401 3490722 api_server.go:52] waiting for apiserver process to appear ...
I0315 07:51:16.677430 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0315 07:51:16.677492 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0315 07:51:16.741347 3490722 cri.go:89] found id: "3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:16.741365 3490722 cri.go:89] found id: "aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:16.741370 3490722 cri.go:89] found id: ""
I0315 07:51:16.741377 3490722 logs.go:276] 2 containers: [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5]
I0315 07:51:16.741429 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.745174 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.748785 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0315 07:51:16.748846 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0315 07:51:13.071627 3500589 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18213-3295134/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-722347:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f -I lz4 -xf /preloaded.tar -C /extractDir: (5.046257196s)
I0315 07:51:13.071659 3500589 kic.go:203] duration metric: took 5.046414304s to extract preloaded images to volume ...
W0315 07:51:13.071823 3500589 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0315 07:51:13.071934 3500589 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0315 07:51:13.126832 3500589 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-722347 --name embed-certs-722347 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-722347 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-722347 --network embed-certs-722347 --ip 192.168.85.2 --volume embed-certs-722347:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f
I0315 07:51:13.492780 3500589 cli_runner.go:164] Run: docker container inspect embed-certs-722347 --format={{.State.Running}}
I0315 07:51:13.517469 3500589 cli_runner.go:164] Run: docker container inspect embed-certs-722347 --format={{.State.Status}}
I0315 07:51:13.537707 3500589 cli_runner.go:164] Run: docker exec embed-certs-722347 stat /var/lib/dpkg/alternatives/iptables
I0315 07:51:13.626528 3500589 oci.go:144] the created container "embed-certs-722347" has a running status.
I0315 07:51:13.626554 3500589 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa...
I0315 07:51:14.312982 3500589 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0315 07:51:14.344591 3500589 cli_runner.go:164] Run: docker container inspect embed-certs-722347 --format={{.State.Status}}
I0315 07:51:14.368961 3500589 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0315 07:51:14.368984 3500589 kic_runner.go:114] Args: [docker exec --privileged embed-certs-722347 chown docker:docker /home/docker/.ssh/authorized_keys]
I0315 07:51:14.444500 3500589 cli_runner.go:164] Run: docker container inspect embed-certs-722347 --format={{.State.Status}}
I0315 07:51:14.469400 3500589 machine.go:94] provisionDockerMachine start ...
I0315 07:51:14.469497 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:14.499265 3500589 main.go:141] libmachine: Using SSH client type: native
I0315 07:51:14.499547 3500589 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36985 <nil> <nil>}
I0315 07:51:14.499558 3500589 main.go:141] libmachine: About to run SSH command:
hostname
I0315 07:51:14.651562 3500589 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-722347
I0315 07:51:14.651586 3500589 ubuntu.go:169] provisioning hostname "embed-certs-722347"
I0315 07:51:14.651653 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:14.683504 3500589 main.go:141] libmachine: Using SSH client type: native
I0315 07:51:14.683752 3500589 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36985 <nil> <nil>}
I0315 07:51:14.683770 3500589 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-722347 && echo "embed-certs-722347" | sudo tee /etc/hostname
I0315 07:51:14.845063 3500589 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-722347
I0315 07:51:14.845199 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:14.864555 3500589 main.go:141] libmachine: Using SSH client type: native
I0315 07:51:14.864788 3500589 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 36985 <nil> <nil>}
I0315 07:51:14.864805 3500589 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-722347' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-722347/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-722347' | sudo tee -a /etc/hosts;
fi
fi
I0315 07:51:15.034245 3500589 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0315 07:51:15.034315 3500589 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18213-3295134/.minikube CaCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18213-3295134/.minikube}
I0315 07:51:15.034354 3500589 ubuntu.go:177] setting up certificates
I0315 07:51:15.034377 3500589 provision.go:84] configureAuth start
I0315 07:51:15.034460 3500589 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-722347
I0315 07:51:15.059134 3500589 provision.go:143] copyHostCerts
I0315 07:51:15.059215 3500589 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem, removing ...
I0315 07:51:15.059233 3500589 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem
I0315 07:51:15.059318 3500589 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.pem (1078 bytes)
I0315 07:51:15.059438 3500589 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem, removing ...
I0315 07:51:15.059449 3500589 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem
I0315 07:51:15.059485 3500589 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/cert.pem (1123 bytes)
I0315 07:51:15.059544 3500589 exec_runner.go:144] found /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem, removing ...
I0315 07:51:15.059556 3500589 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem
I0315 07:51:15.059583 3500589 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18213-3295134/.minikube/key.pem (1679 bytes)
I0315 07:51:15.059634 3500589 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem org=jenkins.embed-certs-722347 san=[127.0.0.1 192.168.85.2 embed-certs-722347 localhost minikube]
I0315 07:51:15.549067 3500589 provision.go:177] copyRemoteCerts
I0315 07:51:15.549135 3500589 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0315 07:51:15.549199 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:15.565719 3500589 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36985 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa Username:docker}
I0315 07:51:15.668354 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0315 07:51:15.699272 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server.pem --> /etc/docker/server.pem (1224 bytes)
I0315 07:51:15.723844 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0315 07:51:15.748402 3500589 provision.go:87] duration metric: took 713.997854ms to configureAuth
I0315 07:51:15.748427 3500589 ubuntu.go:193] setting minikube options for container-runtime
I0315 07:51:15.748604 3500589 config.go:182] Loaded profile config "embed-certs-722347": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.28.4
I0315 07:51:15.748611 3500589 machine.go:97] duration metric: took 1.279194566s to provisionDockerMachine
I0315 07:51:15.748618 3500589 client.go:171] duration metric: took 8.555168333s to LocalClient.Create
I0315 07:51:15.748631 3500589 start.go:167] duration metric: took 8.555227343s to libmachine.API.Create "embed-certs-722347"
I0315 07:51:15.748638 3500589 start.go:293] postStartSetup for "embed-certs-722347" (driver="docker")
I0315 07:51:15.748647 3500589 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0315 07:51:15.748701 3500589 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0315 07:51:15.748750 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:15.765596 3500589 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36985 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa Username:docker}
I0315 07:51:15.872379 3500589 ssh_runner.go:195] Run: cat /etc/os-release
I0315 07:51:15.875630 3500589 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0315 07:51:15.875716 3500589 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0315 07:51:15.875739 3500589 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0315 07:51:15.875747 3500589 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0315 07:51:15.875804 3500589 filesync.go:126] Scanning /home/jenkins/minikube-integration/18213-3295134/.minikube/addons for local assets ...
I0315 07:51:15.875864 3500589 filesync.go:126] Scanning /home/jenkins/minikube-integration/18213-3295134/.minikube/files for local assets ...
I0315 07:51:15.875952 3500589 filesync.go:149] local asset: /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem -> 33005502.pem in /etc/ssl/certs
I0315 07:51:15.876063 3500589 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0315 07:51:15.884721 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem --> /etc/ssl/certs/33005502.pem (1708 bytes)
I0315 07:51:15.909572 3500589 start.go:296] duration metric: took 160.919888ms for postStartSetup
I0315 07:51:15.909996 3500589 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-722347
I0315 07:51:15.932628 3500589 profile.go:142] Saving config to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/config.json ...
I0315 07:51:15.932913 3500589 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0315 07:51:15.933005 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:15.948964 3500589 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36985 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa Username:docker}
I0315 07:51:16.044433 3500589 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0315 07:51:16.049248 3500589 start.go:128] duration metric: took 8.859834932s to createHost
I0315 07:51:16.049271 3500589 start.go:83] releasing machines lock for "embed-certs-722347", held for 8.860000172s
I0315 07:51:16.049370 3500589 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-722347
I0315 07:51:16.067927 3500589 ssh_runner.go:195] Run: cat /version.json
I0315 07:51:16.067987 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:16.068048 3500589 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0315 07:51:16.068116 3500589 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-722347
I0315 07:51:16.092329 3500589 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36985 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa Username:docker}
I0315 07:51:16.104616 3500589 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:36985 SSHKeyPath:/home/jenkins/minikube-integration/18213-3295134/.minikube/machines/embed-certs-722347/id_rsa Username:docker}
I0315 07:51:16.195662 3500589 ssh_runner.go:195] Run: systemctl --version
I0315 07:51:16.324627 3500589 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0315 07:51:16.329572 3500589 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0315 07:51:16.357026 3500589 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0315 07:51:16.357117 3500589 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0315 07:51:16.388919 3500589 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0315 07:51:16.388942 3500589 start.go:494] detecting cgroup driver to use...
I0315 07:51:16.388976 3500589 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0315 07:51:16.389025 3500589 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0315 07:51:16.401311 3500589 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0315 07:51:16.412839 3500589 docker.go:217] disabling cri-docker service (if available) ...
I0315 07:51:16.412904 3500589 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0315 07:51:16.427040 3500589 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0315 07:51:16.442013 3500589 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0315 07:51:16.526842 3500589 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0315 07:51:16.623457 3500589 docker.go:233] disabling docker service ...
I0315 07:51:16.623576 3500589 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0315 07:51:16.646017 3500589 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0315 07:51:16.658104 3500589 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0315 07:51:16.781907 3500589 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0315 07:51:16.882692 3500589 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0315 07:51:16.897241 3500589 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0315 07:51:16.919537 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I0315 07:51:16.932044 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0315 07:51:16.945965 3500589 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0315 07:51:16.946099 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0315 07:51:16.959418 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0315 07:51:16.973939 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0315 07:51:16.992629 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0315 07:51:17.004184 3500589 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0315 07:51:17.024960 3500589 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0315 07:51:17.035748 3500589 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0315 07:51:17.051823 3500589 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0315 07:51:17.065911 3500589 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0315 07:51:17.202561 3500589 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0315 07:51:17.400846 3500589 start.go:541] Will wait 60s for socket path /run/containerd/containerd.sock
I0315 07:51:17.400957 3500589 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0315 07:51:17.409572 3500589 start.go:562] Will wait 60s for crictl version
I0315 07:51:17.409712 3500589 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.413440 3500589 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0315 07:51:17.473401 3500589 start.go:578] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.28
RuntimeApiVersion: v1
I0315 07:51:17.473492 3500589 ssh_runner.go:195] Run: containerd --version
I0315 07:51:17.496872 3500589 ssh_runner.go:195] Run: containerd --version
I0315 07:51:17.527242 3500589 out.go:177] * Preparing Kubernetes v1.28.4 on containerd 1.6.28 ...
I0315 07:51:17.529470 3500589 cli_runner.go:164] Run: docker network inspect embed-certs-722347 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0315 07:51:17.547823 3500589 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0315 07:51:17.551847 3500589 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0315 07:51:17.564343 3500589 kubeadm.go:877] updating cluster {Name:embed-certs-722347 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-722347 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:
false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0315 07:51:17.564458 3500589 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime containerd
I0315 07:51:17.564516 3500589 ssh_runner.go:195] Run: sudo crictl images --output json
I0315 07:51:17.613791 3500589 containerd.go:612] all images are preloaded for containerd runtime.
I0315 07:51:17.613812 3500589 containerd.go:519] Images already preloaded, skipping extraction
I0315 07:51:17.613881 3500589 ssh_runner.go:195] Run: sudo crictl images --output json
I0315 07:51:17.675288 3500589 containerd.go:612] all images are preloaded for containerd runtime.
I0315 07:51:17.675316 3500589 cache_images.go:84] Images are preloaded, skipping loading
I0315 07:51:17.675338 3500589 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.28.4 containerd true true} ...
I0315 07:51:17.675437 3500589 kubeadm.go:940] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.28.4/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-722347 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.28.4 ClusterName:embed-certs-722347 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0315 07:51:17.675500 3500589 ssh_runner.go:195] Run: sudo crictl info
I0315 07:51:17.742197 3500589 cni.go:84] Creating CNI manager for ""
I0315 07:51:17.742217 3500589 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0315 07:51:17.742226 3500589 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0315 07:51:17.742251 3500589 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.28.4 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-722347 NodeName:embed-certs-722347 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0315 07:51:17.742391 3500589 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-722347"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.28.4
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0315 07:51:17.742454 3500589 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.4
I0315 07:51:17.752517 3500589 binaries.go:44] Found k8s binaries, skipping transfer
I0315 07:51:17.752583 3500589 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0315 07:51:17.763155 3500589 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I0315 07:51:17.788503 3500589 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0315 07:51:17.820532 3500589 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I0315 07:51:17.843460 3500589 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0315 07:51:17.847660 3500589 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0315 07:51:17.859015 3500589 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0315 07:51:17.991630 3500589 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0315 07:51:18.010954 3500589 certs.go:68] Setting up /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347 for IP: 192.168.85.2
I0315 07:51:18.010976 3500589 certs.go:194] generating shared ca certs ...
I0315 07:51:18.010994 3500589 certs.go:226] acquiring lock for ca certs: {Name:mk9abb58e338d3f021292a49b0c7ea22df42932a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:18.011226 3500589 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.key
I0315 07:51:18.011268 3500589 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.key
I0315 07:51:18.011276 3500589 certs.go:256] generating profile certs ...
I0315 07:51:18.011336 3500589 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.key
I0315 07:51:18.011348 3500589 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.crt with IP's: []
I0315 07:51:18.248120 3500589 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.crt ...
I0315 07:51:18.248188 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.crt: {Name:mk7afe9d378745aa9b6d80e417cd8b37013d316a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:18.249553 3500589 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.key ...
I0315 07:51:18.249620 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/client.key: {Name:mkd9d15459a043e8e101b1426ba0a8d406e16880 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:18.249818 3500589 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key.6fbee632
I0315 07:51:18.249861 3500589 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt.6fbee632 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.85.2]
I0315 07:51:19.483994 3500589 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt.6fbee632 ...
I0315 07:51:19.484077 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt.6fbee632: {Name:mkff8d19ec8ba146511eb0d74344bce365a04de4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:19.484870 3500589 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key.6fbee632 ...
I0315 07:51:19.484927 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key.6fbee632: {Name:mk75d521a12e8526d566ced7b5778c46290d47f5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:19.485441 3500589 certs.go:381] copying /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt.6fbee632 -> /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt
I0315 07:51:19.485590 3500589 certs.go:385] copying /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key.6fbee632 -> /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key
I0315 07:51:19.485699 3500589 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.key
I0315 07:51:19.485743 3500589 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.crt with IP's: []
I0315 07:51:19.838076 3500589 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.crt ...
I0315 07:51:19.838111 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.crt: {Name:mk16b5ee09b205fa7f0a43fa373a056781ff9234 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:19.838809 3500589 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.key ...
I0315 07:51:19.838835 3500589 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.key: {Name:mk3f05faac83349e1c11e30b10ef096b2f341dd9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0315 07:51:19.840144 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550.pem (1338 bytes)
W0315 07:51:19.840199 3500589 certs.go:480] ignoring /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550_empty.pem, impossibly tiny 0 bytes
I0315 07:51:19.840214 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca-key.pem (1675 bytes)
I0315 07:51:19.840240 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/ca.pem (1078 bytes)
I0315 07:51:19.840267 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/cert.pem (1123 bytes)
I0315 07:51:19.840303 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/key.pem (1679 bytes)
I0315 07:51:19.840350 3500589 certs.go:484] found cert: /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem (1708 bytes)
I0315 07:51:19.840977 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0315 07:51:19.867853 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0315 07:51:19.895744 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0315 07:51:19.927264 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0315 07:51:19.957760 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0315 07:51:19.986080 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0315 07:51:20.018132 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0315 07:51:20.048497 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/profiles/embed-certs-722347/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0315 07:51:20.076603 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0315 07:51:20.105826 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/certs/3300550.pem --> /usr/share/ca-certificates/3300550.pem (1338 bytes)
I0315 07:51:20.137217 3500589 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18213-3295134/.minikube/files/etc/ssl/certs/33005502.pem --> /usr/share/ca-certificates/33005502.pem (1708 bytes)
I0315 07:51:20.162433 3500589 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0315 07:51:20.183501 3500589 ssh_runner.go:195] Run: openssl version
I0315 07:51:20.189913 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0315 07:51:20.200753 3500589 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0315 07:51:20.204637 3500589 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 15 07:01 /usr/share/ca-certificates/minikubeCA.pem
I0315 07:51:20.204747 3500589 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0315 07:51:20.212221 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0315 07:51:20.222138 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3300550.pem && ln -fs /usr/share/ca-certificates/3300550.pem /etc/ssl/certs/3300550.pem"
I0315 07:51:20.232102 3500589 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3300550.pem
I0315 07:51:20.235712 3500589 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 15 07:07 /usr/share/ca-certificates/3300550.pem
I0315 07:51:20.235786 3500589 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3300550.pem
I0315 07:51:20.242713 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/3300550.pem /etc/ssl/certs/51391683.0"
I0315 07:51:20.261609 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/33005502.pem && ln -fs /usr/share/ca-certificates/33005502.pem /etc/ssl/certs/33005502.pem"
I0315 07:51:20.270935 3500589 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/33005502.pem
I0315 07:51:20.274402 3500589 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 15 07:07 /usr/share/ca-certificates/33005502.pem
I0315 07:51:20.274466 3500589 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/33005502.pem
I0315 07:51:20.281512 3500589 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/33005502.pem /etc/ssl/certs/3ec20f2e.0"
I0315 07:51:20.291207 3500589 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0315 07:51:20.294451 3500589 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0315 07:51:20.294553 3500589 kubeadm.go:391] StartCluster: {Name:embed-certs-722347 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:embed-certs-722347 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fal
se CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0315 07:51:20.294643 3500589 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0315 07:51:20.294706 3500589 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0315 07:51:20.344310 3500589 cri.go:89] found id: ""
I0315 07:51:20.344414 3500589 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0315 07:51:20.353344 3500589 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0315 07:51:20.362464 3500589 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver
I0315 07:51:20.362532 3500589 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0315 07:51:20.372499 3500589 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0315 07:51:20.372519 3500589 kubeadm.go:156] found existing configuration files:
I0315 07:51:20.372582 3500589 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0315 07:51:20.382458 3500589 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0315 07:51:20.382530 3500589 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0315 07:51:20.391163 3500589 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0315 07:51:20.401563 3500589 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0315 07:51:20.401656 3500589 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0315 07:51:20.409967 3500589 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0315 07:51:20.419026 3500589 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0315 07:51:20.419116 3500589 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0315 07:51:20.427695 3500589 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0315 07:51:20.436853 3500589 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0315 07:51:20.436919 3500589 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0315 07:51:20.445560 3500589 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.4:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0315 07:51:20.499455 3500589 kubeadm.go:309] [init] Using Kubernetes version: v1.28.4
I0315 07:51:20.499572 3500589 kubeadm.go:309] [preflight] Running pre-flight checks
I0315 07:51:20.559228 3500589 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:
I0315 07:51:20.559335 3500589 kubeadm.go:309] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1055-aws[0m
I0315 07:51:20.559388 3500589 kubeadm.go:309] [0;37mOS[0m: [0;32mLinux[0m
I0315 07:51:20.559463 3500589 kubeadm.go:309] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0315 07:51:20.559533 3500589 kubeadm.go:309] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0315 07:51:20.559598 3500589 kubeadm.go:309] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0315 07:51:20.559656 3500589 kubeadm.go:309] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0315 07:51:20.559774 3500589 kubeadm.go:309] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0315 07:51:20.559882 3500589 kubeadm.go:309] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0315 07:51:20.559962 3500589 kubeadm.go:309] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0315 07:51:20.560050 3500589 kubeadm.go:309] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0315 07:51:20.560145 3500589 kubeadm.go:309] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0315 07:51:20.635853 3500589 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0315 07:51:20.636023 3500589 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0315 07:51:20.636163 3500589 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0315 07:51:20.873186 3500589 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0315 07:51:16.809837 3490722 cri.go:89] found id: "3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:16.809901 3490722 cri.go:89] found id: "288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:16.809917 3490722 cri.go:89] found id: ""
I0315 07:51:16.809925 3490722 logs.go:276] 2 containers: [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf]
I0315 07:51:16.809988 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.821230 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.832434 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0315 07:51:16.832513 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0315 07:51:16.893799 3490722 cri.go:89] found id: "9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:16.893823 3490722 cri.go:89] found id: "7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:16.893828 3490722 cri.go:89] found id: ""
I0315 07:51:16.893836 3490722 logs.go:276] 2 containers: [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc]
I0315 07:51:16.893889 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.898028 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.902368 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0315 07:51:16.902437 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0315 07:51:16.958613 3490722 cri.go:89] found id: "19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:16.958636 3490722 cri.go:89] found id: "eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:16.958642 3490722 cri.go:89] found id: ""
I0315 07:51:16.958649 3490722 logs.go:276] 2 containers: [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d]
I0315 07:51:16.958703 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.962711 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:16.966605 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0315 07:51:16.966681 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0315 07:51:17.016946 3490722 cri.go:89] found id: "24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:17.016970 3490722 cri.go:89] found id: "1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:17.016976 3490722 cri.go:89] found id: ""
I0315 07:51:17.016985 3490722 logs.go:276] 2 containers: [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439]
I0315 07:51:17.017041 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.020761 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.027026 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0315 07:51:17.027194 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0315 07:51:17.081451 3490722 cri.go:89] found id: "39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:17.081475 3490722 cri.go:89] found id: "6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:17.081480 3490722 cri.go:89] found id: ""
I0315 07:51:17.081487 3490722 logs.go:276] 2 containers: [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d]
I0315 07:51:17.081544 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.085402 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.088913 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0315 07:51:17.088999 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0315 07:51:17.152035 3490722 cri.go:89] found id: "d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:17.152107 3490722 cri.go:89] found id: "7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:17.152125 3490722 cri.go:89] found id: ""
I0315 07:51:17.152146 3490722 logs.go:276] 2 containers: [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e]
I0315 07:51:17.152229 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.156132 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.159619 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0315 07:51:17.159727 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0315 07:51:17.208743 3490722 cri.go:89] found id: "141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:17.208812 3490722 cri.go:89] found id: ""
I0315 07:51:17.208834 3490722 logs.go:276] 1 containers: [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c]
I0315 07:51:17.208916 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.212643 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0315 07:51:17.212756 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0315 07:51:17.276269 3490722 cri.go:89] found id: "ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:17.276352 3490722 cri.go:89] found id: "4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:17.276372 3490722 cri.go:89] found id: ""
I0315 07:51:17.276394 3490722 logs.go:276] 2 containers: [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2]
I0315 07:51:17.276485 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.281157 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:17.285880 3490722 logs.go:123] Gathering logs for kube-apiserver [aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5] ...
I0315 07:51:17.285975 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:17.383966 3490722 logs.go:123] Gathering logs for etcd [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe] ...
I0315 07:51:17.383996 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:17.452945 3490722 logs.go:123] Gathering logs for etcd [288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf] ...
I0315 07:51:17.453085 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:17.518825 3490722 logs.go:123] Gathering logs for kube-proxy [1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439] ...
I0315 07:51:17.518972 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:17.582975 3490722 logs.go:123] Gathering logs for kube-controller-manager [6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d] ...
I0315 07:51:17.582999 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:17.721400 3490722 logs.go:123] Gathering logs for kindnet [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842] ...
I0315 07:51:17.721438 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:17.782289 3490722 logs.go:123] Gathering logs for kubelet ...
I0315 07:51:17.782317 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0315 07:51:17.844998 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151553 663 reflector.go:138] object-"kube-system"/"metrics-server-token-h2gnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-h2gnq" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845272 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151674 663 reflector.go:138] object-"default"/"default-token-82cb9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-82cb9" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845526 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151731 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845762 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151795 663 reflector.go:138] object-"kube-system"/"coredns-token-vw5pl": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-vw5pl" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.845999 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151873 663 reflector.go:138] object-"kube-system"/"kindnet-token-jrrqr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jrrqr" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846238 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151952 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-vqzhx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vqzhx" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846467 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152014 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.846715 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152079 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-hnqzn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-hnqzn" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:17.857570 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:58 old-k8s-version-591842 kubelet[663]: E0315 07:45:58.854597 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.857857 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:59 old-k8s-version-591842 kubelet[663]: E0315 07:45:59.802434 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.861219 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:14 old-k8s-version-591842 kubelet[663]: E0315 07:46:14.653468 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.863383 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:20 old-k8s-version-591842 kubelet[663]: E0315 07:46:20.888372 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.863745 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:21 old-k8s-version-591842 kubelet[663]: E0315 07:46:21.893270 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.864106 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:22 old-k8s-version-591842 kubelet[663]: E0315 07:46:22.992818 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.864314 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:25 old-k8s-version-591842 kubelet[663]: E0315 07:46:25.647507 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.865123 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:30 old-k8s-version-591842 kubelet[663]: E0315 07:46:30.928822 663 pod_workers.go:191] Error syncing pod 1b02bdb3-5934-4002-980c-769d1de68357 ("storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"
W0315 07:51:17.865704 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:35 old-k8s-version-591842 kubelet[663]: E0315 07:46:35.946453 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.868835 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:37 old-k8s-version-591842 kubelet[663]: E0315 07:46:37.659626 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.869661 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:42 old-k8s-version-591842 kubelet[663]: E0315 07:46:42.993141 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.869869 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:48 old-k8s-version-591842 kubelet[663]: E0315 07:46:48.651057 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.870811 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:58 old-k8s-version-591842 kubelet[663]: E0315 07:46:58.015895 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.871197 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:02 old-k8s-version-591842 kubelet[663]: E0315 07:47:02.992859 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.871475 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:03 old-k8s-version-591842 kubelet[663]: E0315 07:47:03.649062 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.871720 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:15 old-k8s-version-591842 kubelet[663]: E0315 07:47:15.644773 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.872103 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:16 old-k8s-version-591842 kubelet[663]: E0315 07:47:16.645024 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.874664 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:29 old-k8s-version-591842 kubelet[663]: E0315 07:47:29.656112 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.875037 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:30 old-k8s-version-591842 kubelet[663]: E0315 07:47:30.644347 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.875270 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:42 old-k8s-version-591842 kubelet[663]: E0315 07:47:42.645173 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.875894 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:45 old-k8s-version-591842 kubelet[663]: E0315 07:47:45.161682 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.876256 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:52 old-k8s-version-591842 kubelet[663]: E0315 07:47:52.992815 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.876463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:54 old-k8s-version-591842 kubelet[663]: E0315 07:47:54.645803 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.876813 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.644489 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.877020 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.645236 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.877369 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.644500 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.877577 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.645259 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.877786 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:29 old-k8s-version-591842 kubelet[663]: E0315 07:48:29.644702 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.878133 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:33 old-k8s-version-591842 kubelet[663]: E0315 07:48:33.645925 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.878366 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:42 old-k8s-version-591842 kubelet[663]: E0315 07:48:42.644688 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.878717 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:45 old-k8s-version-591842 kubelet[663]: E0315 07:48:45.644849 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.881377 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:57 old-k8s-version-591842 kubelet[663]: E0315 07:48:57.654804 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:17.881741 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:59 old-k8s-version-591842 kubelet[663]: E0315 07:48:59.644341 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.881947 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:09 old-k8s-version-591842 kubelet[663]: E0315 07:49:09.653348 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.882559 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:14 old-k8s-version-591842 kubelet[663]: E0315 07:49:14.388467 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.882911 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:22 old-k8s-version-591842 kubelet[663]: E0315 07:49:22.993103 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.883162 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:23 old-k8s-version-591842 kubelet[663]: E0315 07:49:23.645937 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.883370 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:35 old-k8s-version-591842 kubelet[663]: E0315 07:49:35.645912 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.883715 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:36 old-k8s-version-591842 kubelet[663]: E0315 07:49:36.644319 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.883929 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:47 old-k8s-version-591842 kubelet[663]: E0315 07:49:47.658879 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.884276 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:50 old-k8s-version-591842 kubelet[663]: E0315 07:49:50.644297 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.884485 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:02 old-k8s-version-591842 kubelet[663]: E0315 07:50:02.644922 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.884840 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:05 old-k8s-version-591842 kubelet[663]: E0315 07:50:05.644906 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.885212 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.645285 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.885484 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.647267 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.889275 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: E0315 07:50:28.644332 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.889531 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:29 old-k8s-version-591842 kubelet[663]: E0315 07:50:29.645326 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.889739 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:42 old-k8s-version-591842 kubelet[663]: E0315 07:50:42.644973 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.890100 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.890309 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:17.890655 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.891006 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:17.891245 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0315 07:51:17.891273 3490722 logs.go:123] Gathering logs for dmesg ...
I0315 07:51:17.891305 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0315 07:51:17.929063 3490722 logs.go:123] Gathering logs for kindnet [7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e] ...
I0315 07:51:17.929137 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:17.993303 3490722 logs.go:123] Gathering logs for kube-scheduler [eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d] ...
I0315 07:51:17.993327 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:18.145483 3490722 logs.go:123] Gathering logs for container status ...
I0315 07:51:18.145518 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0315 07:51:18.232250 3490722 logs.go:123] Gathering logs for kube-apiserver [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465] ...
I0315 07:51:18.232279 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:18.307131 3490722 logs.go:123] Gathering logs for coredns [7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc] ...
I0315 07:51:18.307162 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:18.372534 3490722 logs.go:123] Gathering logs for kubernetes-dashboard [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c] ...
I0315 07:51:18.372563 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:18.423802 3490722 logs.go:123] Gathering logs for coredns [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e] ...
I0315 07:51:18.423835 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:18.484538 3490722 logs.go:123] Gathering logs for kube-controller-manager [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2] ...
I0315 07:51:18.484567 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:18.734982 3490722 logs.go:123] Gathering logs for kube-proxy [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3] ...
I0315 07:51:18.735022 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:18.989000 3490722 logs.go:123] Gathering logs for storage-provisioner [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803] ...
I0315 07:51:18.989036 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:19.097844 3490722 logs.go:123] Gathering logs for storage-provisioner [4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2] ...
I0315 07:51:19.097876 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:19.152808 3490722 logs.go:123] Gathering logs for containerd ...
I0315 07:51:19.152845 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0315 07:51:19.226776 3490722 logs.go:123] Gathering logs for describe nodes ...
I0315 07:51:19.226813 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0315 07:51:19.465101 3490722 logs.go:123] Gathering logs for kube-scheduler [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423] ...
I0315 07:51:19.465178 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:19.520188 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:19.520213 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0315 07:51:19.520268 3490722 out.go:239] X Problems detected in kubelet:
W0315 07:51:19.520277 3490722 out.go:239] Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520285 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:19.520293 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520302 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:19.520311 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0315 07:51:19.520322 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:19.520329 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:20.877620 3500589 out.go:204] - Generating certificates and keys ...
I0315 07:51:20.877730 3500589 kubeadm.go:309] [certs] Using existing ca certificate authority
I0315 07:51:20.877819 3500589 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0315 07:51:21.260611 3500589 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0315 07:51:21.550266 3500589 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0315 07:51:22.311695 3500589 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0315 07:51:22.560765 3500589 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0315 07:51:23.248609 3500589 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0315 07:51:23.248946 3500589 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [embed-certs-722347 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0315 07:51:24.037075 3500589 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0315 07:51:24.037433 3500589 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-722347 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0315 07:51:25.143335 3500589 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0315 07:51:26.413409 3500589 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0315 07:51:27.100054 3500589 kubeadm.go:309] [certs] Generating "sa" key and public key
I0315 07:51:27.100355 3500589 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0315 07:51:27.268640 3500589 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0315 07:51:27.893237 3500589 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0315 07:51:28.267254 3500589 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0315 07:51:28.997024 3500589 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0315 07:51:28.997796 3500589 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0315 07:51:29.000556 3500589 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0315 07:51:29.521502 3490722 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0315 07:51:29.538525 3490722 api_server.go:72] duration metric: took 5m49.79464477s to wait for apiserver process to appear ...
I0315 07:51:29.538551 3490722 api_server.go:88] waiting for apiserver healthz status ...
I0315 07:51:29.538596 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0315 07:51:29.538676 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0315 07:51:29.599859 3490722 cri.go:89] found id: "3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:29.599884 3490722 cri.go:89] found id: "aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:29.599889 3490722 cri.go:89] found id: ""
I0315 07:51:29.599897 3490722 logs.go:276] 2 containers: [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5]
I0315 07:51:29.599971 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.605961 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.610920 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0315 07:51:29.611054 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0315 07:51:29.685499 3490722 cri.go:89] found id: "3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:29.685526 3490722 cri.go:89] found id: "288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:29.685530 3490722 cri.go:89] found id: ""
I0315 07:51:29.685537 3490722 logs.go:276] 2 containers: [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf]
I0315 07:51:29.685610 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.690445 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.695366 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0315 07:51:29.695452 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0315 07:51:29.771248 3490722 cri.go:89] found id: "9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:29.771267 3490722 cri.go:89] found id: "7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:29.771272 3490722 cri.go:89] found id: ""
I0315 07:51:29.771279 3490722 logs.go:276] 2 containers: [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc]
I0315 07:51:29.771337 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.776267 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.793698 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0315 07:51:29.793769 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0315 07:51:29.845789 3490722 cri.go:89] found id: "19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:29.845851 3490722 cri.go:89] found id: "eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:29.845878 3490722 cri.go:89] found id: ""
I0315 07:51:29.845898 3490722 logs.go:276] 2 containers: [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d]
I0315 07:51:29.845985 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.851019 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.855662 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0315 07:51:29.855846 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0315 07:51:29.914978 3490722 cri.go:89] found id: "24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:29.915050 3490722 cri.go:89] found id: "1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:29.915095 3490722 cri.go:89] found id: ""
I0315 07:51:29.915121 3490722 logs.go:276] 2 containers: [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439]
I0315 07:51:29.915205 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.923722 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.927822 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0315 07:51:29.927942 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0315 07:51:29.989735 3490722 cri.go:89] found id: "39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:29.989797 3490722 cri.go:89] found id: "6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:29.989825 3490722 cri.go:89] found id: ""
I0315 07:51:29.989846 3490722 logs.go:276] 2 containers: [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d]
I0315 07:51:29.989933 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.994641 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:29.998628 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0315 07:51:29.998750 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0315 07:51:30.121952 3490722 cri.go:89] found id: "d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:30.122032 3490722 cri.go:89] found id: "7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:30.122054 3490722 cri.go:89] found id: ""
I0315 07:51:30.122083 3490722 logs.go:276] 2 containers: [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e]
I0315 07:51:30.122196 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.127774 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.144595 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0315 07:51:30.144766 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0315 07:51:30.219866 3490722 cri.go:89] found id: "141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:30.219939 3490722 cri.go:89] found id: ""
I0315 07:51:30.219962 3490722 logs.go:276] 1 containers: [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c]
I0315 07:51:30.220076 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.226314 3490722 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0315 07:51:30.226500 3490722 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0315 07:51:30.285822 3490722 cri.go:89] found id: "ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:30.285848 3490722 cri.go:89] found id: "4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:30.285854 3490722 cri.go:89] found id: ""
I0315 07:51:30.285862 3490722 logs.go:276] 2 containers: [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2]
I0315 07:51:30.285924 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.291346 3490722 ssh_runner.go:195] Run: which crictl
I0315 07:51:30.296522 3490722 logs.go:123] Gathering logs for kubernetes-dashboard [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c] ...
I0315 07:51:30.296594 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c"
I0315 07:51:30.367022 3490722 logs.go:123] Gathering logs for storage-provisioner [4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2] ...
I0315 07:51:30.367147 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2"
I0315 07:51:30.420538 3490722 logs.go:123] Gathering logs for containerd ...
I0315 07:51:30.420567 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0315 07:51:30.496522 3490722 logs.go:123] Gathering logs for container status ...
I0315 07:51:30.496650 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0315 07:51:30.579010 3490722 logs.go:123] Gathering logs for kube-scheduler [eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d] ...
I0315 07:51:30.579049 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d"
I0315 07:51:30.630592 3490722 logs.go:123] Gathering logs for kube-proxy [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3] ...
I0315 07:51:30.630626 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3"
I0315 07:51:30.678109 3490722 logs.go:123] Gathering logs for kindnet [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842] ...
I0315 07:51:30.678138 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842"
I0315 07:51:30.723399 3490722 logs.go:123] Gathering logs for kube-apiserver [aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5] ...
I0315 07:51:30.723428 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5"
I0315 07:51:30.797968 3490722 logs.go:123] Gathering logs for dmesg ...
I0315 07:51:30.798005 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0315 07:51:30.816622 3490722 logs.go:123] Gathering logs for describe nodes ...
I0315 07:51:30.816653 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0315 07:51:30.972164 3490722 logs.go:123] Gathering logs for kube-apiserver [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465] ...
I0315 07:51:30.972197 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465"
I0315 07:51:31.050833 3490722 logs.go:123] Gathering logs for kindnet [7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e] ...
I0315 07:51:31.050875 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e"
I0315 07:51:31.123735 3490722 logs.go:123] Gathering logs for storage-provisioner [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803] ...
I0315 07:51:31.123775 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803"
I0315 07:51:31.190207 3490722 logs.go:123] Gathering logs for coredns [7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc] ...
I0315 07:51:31.190236 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc"
I0315 07:51:31.249826 3490722 logs.go:123] Gathering logs for kube-controller-manager [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2] ...
I0315 07:51:31.249855 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2"
I0315 07:51:31.354117 3490722 logs.go:123] Gathering logs for kube-controller-manager [6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d] ...
I0315 07:51:31.354155 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d"
I0315 07:51:31.441541 3490722 logs.go:123] Gathering logs for coredns [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e] ...
I0315 07:51:31.441581 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e"
I0315 07:51:31.515474 3490722 logs.go:123] Gathering logs for kube-scheduler [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423] ...
I0315 07:51:31.515503 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423"
I0315 07:51:31.589337 3490722 logs.go:123] Gathering logs for kube-proxy [1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439] ...
I0315 07:51:31.589367 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439"
I0315 07:51:31.679207 3490722 logs.go:123] Gathering logs for kubelet ...
I0315 07:51:31.679237 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0315 07:51:31.765777 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151553 663 reflector.go:138] object-"kube-system"/"metrics-server-token-h2gnq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-h2gnq" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766009 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151674 663 reflector.go:138] object-"default"/"default-token-82cb9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-82cb9" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766219 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151731 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766444 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151795 663 reflector.go:138] object-"kube-system"/"coredns-token-vw5pl": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-vw5pl" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766666 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151873 663 reflector.go:138] object-"kube-system"/"kindnet-token-jrrqr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jrrqr" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.766881 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.151952 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-vqzhx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-vqzhx" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.767126 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152014 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.767352 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:56 old-k8s-version-591842 kubelet[663]: E0315 07:45:56.152079 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-hnqzn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-hnqzn" is forbidden: User "system:node:old-k8s-version-591842" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-591842' and this object
W0315 07:51:31.777873 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:58 old-k8s-version-591842 kubelet[663]: E0315 07:45:58.854597 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.778081 3490722 logs.go:138] Found kubelet problem: Mar 15 07:45:59 old-k8s-version-591842 kubelet[663]: E0315 07:45:59.802434 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.784967 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:14 old-k8s-version-591842 kubelet[663]: E0315 07:46:14.653468 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.787050 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:20 old-k8s-version-591842 kubelet[663]: E0315 07:46:20.888372 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.788741 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:21 old-k8s-version-591842 kubelet[663]: E0315 07:46:21.893270 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.789087 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:22 old-k8s-version-591842 kubelet[663]: E0315 07:46:22.992818 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.789271 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:25 old-k8s-version-591842 kubelet[663]: E0315 07:46:25.647507 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.790036 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:30 old-k8s-version-591842 kubelet[663]: E0315 07:46:30.928822 663 pod_workers.go:191] Error syncing pod 1b02bdb3-5934-4002-980c-769d1de68357 ("storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(1b02bdb3-5934-4002-980c-769d1de68357)"
W0315 07:51:31.790614 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:35 old-k8s-version-591842 kubelet[663]: E0315 07:46:35.946453 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.793030 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:37 old-k8s-version-591842 kubelet[663]: E0315 07:46:37.659626 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.793817 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:42 old-k8s-version-591842 kubelet[663]: E0315 07:46:42.993141 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.794004 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:48 old-k8s-version-591842 kubelet[663]: E0315 07:46:48.651057 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.794583 3490722 logs.go:138] Found kubelet problem: Mar 15 07:46:58 old-k8s-version-591842 kubelet[663]: E0315 07:46:58.015895 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.794910 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:02 old-k8s-version-591842 kubelet[663]: E0315 07:47:02.992859 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
I0315 07:51:29.002553 3500589 out.go:204] - Booting up control plane ...
I0315 07:51:29.002670 3500589 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0315 07:51:29.009978 3500589 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0315 07:51:29.010073 3500589 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0315 07:51:29.022602 3500589 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0315 07:51:29.023683 3500589 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0315 07:51:29.023922 3500589 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0315 07:51:29.137586 3500589 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
W0315 07:51:31.797506 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:03 old-k8s-version-591842 kubelet[663]: E0315 07:47:03.649062 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.800132 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:15 old-k8s-version-591842 kubelet[663]: E0315 07:47:15.644773 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.800463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:16 old-k8s-version-591842 kubelet[663]: E0315 07:47:16.645024 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.802857 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:29 old-k8s-version-591842 kubelet[663]: E0315 07:47:29.656112 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.803187 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:30 old-k8s-version-591842 kubelet[663]: E0315 07:47:30.644347 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.803376 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:42 old-k8s-version-591842 kubelet[663]: E0315 07:47:42.645173 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.803959 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:45 old-k8s-version-591842 kubelet[663]: E0315 07:47:45.161682 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804281 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:52 old-k8s-version-591842 kubelet[663]: E0315 07:47:52.992815 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804463 3490722 logs.go:138] Found kubelet problem: Mar 15 07:47:54 old-k8s-version-591842 kubelet[663]: E0315 07:47:54.645803 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.804784 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.644489 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.804966 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:06 old-k8s-version-591842 kubelet[663]: E0315 07:48:06.645236 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805298 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.644500 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.805483 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:18 old-k8s-version-591842 kubelet[663]: E0315 07:48:18.645259 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805664 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:29 old-k8s-version-591842 kubelet[663]: E0315 07:48:29.644702 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.805987 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:33 old-k8s-version-591842 kubelet[663]: E0315 07:48:33.645925 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.806168 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:42 old-k8s-version-591842 kubelet[663]: E0315 07:48:42.644688 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.806493 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:45 old-k8s-version-591842 kubelet[663]: E0315 07:48:45.644849 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.808911 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:57 old-k8s-version-591842 kubelet[663]: E0315 07:48:57.654804 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0315 07:51:31.809237 3490722 logs.go:138] Found kubelet problem: Mar 15 07:48:59 old-k8s-version-591842 kubelet[663]: E0315 07:48:59.644341 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.809453 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:09 old-k8s-version-591842 kubelet[663]: E0315 07:49:09.653348 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.810032 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:14 old-k8s-version-591842 kubelet[663]: E0315 07:49:14.388467 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.810354 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:22 old-k8s-version-591842 kubelet[663]: E0315 07:49:22.993103 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.810536 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:23 old-k8s-version-591842 kubelet[663]: E0315 07:49:23.645937 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.810717 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:35 old-k8s-version-591842 kubelet[663]: E0315 07:49:35.645912 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.811038 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:36 old-k8s-version-591842 kubelet[663]: E0315 07:49:36.644319 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.813162 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:47 old-k8s-version-591842 kubelet[663]: E0315 07:49:47.658879 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.813501 3490722 logs.go:138] Found kubelet problem: Mar 15 07:49:50 old-k8s-version-591842 kubelet[663]: E0315 07:49:50.644297 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.813686 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:02 old-k8s-version-591842 kubelet[663]: E0315 07:50:02.644922 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.814013 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:05 old-k8s-version-591842 kubelet[663]: E0315 07:50:05.644906 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.814335 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.645285 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.814518 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.647267 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.814841 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: E0315 07:50:28.644332 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.815023 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:29 old-k8s-version-591842 kubelet[663]: E0315 07:50:29.645326 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816134 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:42 old-k8s-version-591842 kubelet[663]: E0315 07:50:42.644973 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816475 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.816660 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.816996 3490722 logs.go:138] Found kubelet problem: Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.817320 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.817501 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.817684 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.818007 3490722 logs.go:138] Found kubelet problem: Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
I0315 07:51:31.818018 3490722 logs.go:123] Gathering logs for etcd [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe] ...
I0315 07:51:31.818032 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe"
I0315 07:51:31.903966 3490722 logs.go:123] Gathering logs for etcd [288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf] ...
I0315 07:51:31.903997 3490722 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf"
I0315 07:51:31.986316 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:31.986344 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0315 07:51:31.986390 3490722 out.go:239] X Problems detected in kubelet:
W0315 07:51:31.986404 3490722 out.go:239] Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.986414 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
W0315 07:51:31.986428 3490722 out.go:239] Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.986435 3490722 out.go:239] Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0315 07:51:31.986449 3490722 out.go:239] Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
I0315 07:51:31.986457 3490722 out.go:304] Setting ErrFile to fd 2...
I0315 07:51:31.986464 3490722 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0315 07:51:38.141417 3500589 kubeadm.go:309] [apiclient] All control plane components are healthy after 9.004204 seconds
I0315 07:51:38.141546 3500589 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0315 07:51:38.161747 3500589 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0315 07:51:38.689025 3500589 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs
I0315 07:51:38.689223 3500589 kubeadm.go:309] [mark-control-plane] Marking the node embed-certs-722347 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0315 07:51:39.202430 3500589 kubeadm.go:309] [bootstrap-token] Using token: 2gd82e.ximahph8csemur3g
I0315 07:51:39.204554 3500589 out.go:204] - Configuring RBAC rules ...
I0315 07:51:39.204685 3500589 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0315 07:51:39.213025 3500589 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0315 07:51:39.226347 3500589 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0315 07:51:39.230690 3500589 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0315 07:51:39.235258 3500589 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0315 07:51:39.242218 3500589 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0315 07:51:39.256887 3500589 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0315 07:51:39.513255 3500589 kubeadm.go:309] [addons] Applied essential addon: CoreDNS
I0315 07:51:39.626946 3500589 kubeadm.go:309] [addons] Applied essential addon: kube-proxy
I0315 07:51:39.628881 3500589 kubeadm.go:309]
I0315 07:51:39.628951 3500589 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!
I0315 07:51:39.628962 3500589 kubeadm.go:309]
I0315 07:51:39.629037 3500589 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:
I0315 07:51:39.629042 3500589 kubeadm.go:309]
I0315 07:51:39.629066 3500589 kubeadm.go:309] mkdir -p $HOME/.kube
I0315 07:51:39.629123 3500589 kubeadm.go:309] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0315 07:51:39.629172 3500589 kubeadm.go:309] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0315 07:51:39.629176 3500589 kubeadm.go:309]
I0315 07:51:39.629227 3500589 kubeadm.go:309] Alternatively, if you are the root user, you can run:
I0315 07:51:39.629231 3500589 kubeadm.go:309]
I0315 07:51:39.629277 3500589 kubeadm.go:309] export KUBECONFIG=/etc/kubernetes/admin.conf
I0315 07:51:39.629281 3500589 kubeadm.go:309]
I0315 07:51:39.629331 3500589 kubeadm.go:309] You should now deploy a pod network to the cluster.
I0315 07:51:39.629425 3500589 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0315 07:51:39.629492 3500589 kubeadm.go:309] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0315 07:51:39.629496 3500589 kubeadm.go:309]
I0315 07:51:39.629822 3500589 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities
I0315 07:51:39.629903 3500589 kubeadm.go:309] and service account keys on each node and then running the following as root:
I0315 07:51:39.629908 3500589 kubeadm.go:309]
I0315 07:51:39.629988 3500589 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token 2gd82e.ximahph8csemur3g \
I0315 07:51:39.630086 3500589 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:c1e97d56565bc0beab8ad4377b38bf3319ec6c746cc5fae6ed0032cea307c48a \
I0315 07:51:39.630107 3500589 kubeadm.go:309] --control-plane
I0315 07:51:39.630111 3500589 kubeadm.go:309]
I0315 07:51:39.630203 3500589 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:
I0315 07:51:39.630209 3500589 kubeadm.go:309]
I0315 07:51:39.630484 3500589 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token 2gd82e.ximahph8csemur3g \
I0315 07:51:39.630588 3500589 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:c1e97d56565bc0beab8ad4377b38bf3319ec6c746cc5fae6ed0032cea307c48a
I0315 07:51:39.635513 3500589 kubeadm.go:309] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1055-aws\n", err: exit status 1
I0315 07:51:39.635627 3500589 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0315 07:51:39.635972 3500589 cni.go:84] Creating CNI manager for ""
I0315 07:51:39.635985 3500589 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0315 07:51:39.638532 3500589 out.go:177] * Configuring CNI (Container Networking Interface) ...
I0315 07:51:39.640686 3500589 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap
I0315 07:51:39.652601 3500589 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.28.4/kubectl ...
I0315 07:51:39.652620 3500589 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes)
I0315 07:51:39.691276 3500589 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0315 07:51:40.876088 3500589 ssh_runner.go:235] Completed: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: (1.184772671s)
I0315 07:51:40.876122 3500589 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0315 07:51:40.876232 3500589 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0315 07:51:40.876305 3500589 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes embed-certs-722347 minikube.k8s.io/updated_at=2024_03_15T07_51_40_0700 minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=eb91ca077853a04b88fe45b419a329f4c2efcc56 minikube.k8s.io/name=embed-certs-722347 minikube.k8s.io/primary=true
I0315 07:51:41.051910 3500589 ops.go:34] apiserver oom_adj: -16
I0315 07:51:41.052010 3500589 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0315 07:51:41.552087 3500589 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0315 07:51:41.987133 3490722 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0315 07:51:41.998532 3490722 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0315 07:51:42.008114 3490722 out.go:177]
W0315 07:51:42.013913 3490722 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0315 07:51:42.013959 3490722 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0315 07:51:42.013981 3490722 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0315 07:51:42.013988 3490722 out.go:239] *
W0315 07:51:42.015597 3490722 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0315 07:51:42.018503 3490722 out.go:177]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
c8441ee92457c 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 73a8bde3ab233 dashboard-metrics-scraper-8d5bb5db8-2wnk8
ae03a9797792d ba04bb24b9575 5 minutes ago Running storage-provisioner 3 4e15d2dd71844 storage-provisioner
141c1019fad2f 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 f5c3e831f3b38 kubernetes-dashboard-cd95d586-7fkx2
4518d148ada79 ba04bb24b9575 5 minutes ago Exited storage-provisioner 2 4e15d2dd71844 storage-provisioner
9a5842dfbaea4 db91994f4ee8f 5 minutes ago Running coredns 1 07b030b94131a coredns-74ff55c5b-5zhc9
24cc1f1e367cb 25a5233254979 5 minutes ago Running kube-proxy 1 73bb07e1652b8 kube-proxy-pdn2n
d3767d4efec19 4740c1948d3fc 5 minutes ago Running kindnet-cni 1 7f6797fa1f610 kindnet-9mqv4
1cb8a13f25e13 1611cd07b61d5 5 minutes ago Running busybox 1 1cc07abf2b05a busybox
3e9f3d24aea76 05b738aa1bc63 5 minutes ago Running etcd 1 c1db48ada60e0 etcd-old-k8s-version-591842
3aee10dec5ed0 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 6cf2f921c3cdf kube-apiserver-old-k8s-version-591842
39d3647417de7 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 aa4a3ddf943cb kube-controller-manager-old-k8s-version-591842
19906a36a5bc6 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 2745bd156d2e4 kube-scheduler-old-k8s-version-591842
4cf010663380e 1611cd07b61d5 6 minutes ago Exited busybox 0 521cfec301725 busybox
7d25cff184892 db91994f4ee8f 7 minutes ago Exited coredns 0 bf3adcb851ee9 coredns-74ff55c5b-5zhc9
1909102cb4c31 25a5233254979 8 minutes ago Exited kube-proxy 0 5209bf3170a14 kube-proxy-pdn2n
7113762b904f4 4740c1948d3fc 8 minutes ago Exited kindnet-cni 0 c2b86ece2a094 kindnet-9mqv4
6e6c79cda8326 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 25782a2d7ba87 kube-controller-manager-old-k8s-version-591842
eb46f252b98e7 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 cecb29930549c kube-scheduler-old-k8s-version-591842
aa813eda9f1da 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 c3e742b203849 kube-apiserver-old-k8s-version-591842
288c914996f04 05b738aa1bc63 8 minutes ago Exited etcd 0 7616c9e23ef1d etcd-old-k8s-version-591842
==> containerd <==
Mar 15 07:47:29 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:29.653100255Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Mar 15 07:47:29 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:29.654734260Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.647966996Z" level=info msg="CreateContainer within sandbox \"73a8bde3ab233fe4898e2abd1a3f4732846c8aa764491c9f3231263c8457dfe7\" for container &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:4,}"
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.680324893Z" level=info msg="CreateContainer within sandbox \"73a8bde3ab233fe4898e2abd1a3f4732846c8aa764491c9f3231263c8457dfe7\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:4,} returns container id \"ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c\""
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.681219568Z" level=info msg="StartContainer for \"ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c\""
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.745003617Z" level=info msg="StartContainer for \"ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c\" returns successfully"
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.772954298Z" level=info msg="shim disconnected" id=ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.773029931Z" level=warning msg="cleaning up after shim disconnected" id=ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c namespace=k8s.io
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.773041541Z" level=info msg="cleaning up dead shim"
Mar 15 07:47:44 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:44.781705257Z" level=warning msg="cleanup warnings time=\"2024-03-15T07:47:44Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=2970 runtime=io.containerd.runc.v2\n"
Mar 15 07:47:45 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:45.167044881Z" level=info msg="RemoveContainer for \"bbb7bdf5ec489800fb498e39fb48eccfc81007bb9eee3a36f4ab67d9c3a32d30\""
Mar 15 07:47:45 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:47:45.185763466Z" level=info msg="RemoveContainer for \"bbb7bdf5ec489800fb498e39fb48eccfc81007bb9eee3a36f4ab67d9c3a32d30\" returns successfully"
Mar 15 07:48:57 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:48:57.646459680Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:48:57 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:48:57.652857438Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Mar 15 07:48:57 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:48:57.654431899Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.647061699Z" level=info msg="CreateContainer within sandbox \"73a8bde3ab233fe4898e2abd1a3f4732846c8aa764491c9f3231263c8457dfe7\" for container &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,}"
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.662441696Z" level=info msg="CreateContainer within sandbox \"73a8bde3ab233fe4898e2abd1a3f4732846c8aa764491c9f3231263c8457dfe7\" for &ContainerMetadata{Name:dashboard-metrics-scraper,Attempt:5,} returns container id \"c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473\""
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.662855315Z" level=info msg="StartContainer for \"c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473\""
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.727041490Z" level=info msg="StartContainer for \"c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473\" returns successfully"
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.771950617Z" level=info msg="shim disconnected" id=c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.772011744Z" level=warning msg="cleaning up after shim disconnected" id=c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473 namespace=k8s.io
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.772022804Z" level=info msg="cleaning up dead shim"
Mar 15 07:49:13 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:13.780634074Z" level=warning msg="cleanup warnings time=\"2024-03-15T07:49:13Z\" level=info msg=\"starting signal loop\" namespace=k8s.io pid=3225 runtime=io.containerd.runc.v2\n"
Mar 15 07:49:14 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:14.405385669Z" level=info msg="RemoveContainer for \"ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c\""
Mar 15 07:49:14 old-k8s-version-591842 containerd[568]: time="2024-03-15T07:49:14.410990904Z" level=info msg="RemoveContainer for \"ac754e0933e1f658cc96c097e5a23e5e08c460ba63919802a82dcc883556a77c\" returns successfully"
==> coredns [7d25cff18489226da998ecaf53342d2b701293554532a64fa95727207acd81dc] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:37629 - 62494 "HINFO IN 7250847379127815357.2933103892824746515. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.021557872s
==> coredns [9a5842dfbaea4b23c98bf7b430f498be93e7775283a1f0649b405d63584f2d1e] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:42605 - 47529 "HINFO IN 9206751975402680439.6531928222192876371. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.020469972s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0315 07:46:30.600120 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-15 07:46:00.599506861 +0000 UTC m=+0.022877047) (total time: 30.000517334s):
Trace[2019727887]: [30.000517334s] [30.000517334s] END
E0315 07:46:30.600162 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0315 07:46:30.601146 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-15 07:46:00.60072161 +0000 UTC m=+0.024091796) (total time: 30.000406666s):
Trace[939984059]: [30.000406666s] [30.000406666s] END
E0315 07:46:30.601165 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0315 07:46:30.601932 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-15 07:46:00.601617441 +0000 UTC m=+0.024987635) (total time: 30.000299387s):
Trace[1474941318]: [30.000299387s] [30.000299387s] END
E0315 07:46:30.601950 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> describe nodes <==
Name: old-k8s-version-591842
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-591842
kubernetes.io/os=linux
minikube.k8s.io/commit=eb91ca077853a04b88fe45b419a329f4c2efcc56
minikube.k8s.io/name=old-k8s-version-591842
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_03_15T07_42_57_0700
minikube.k8s.io/version=v1.32.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Fri, 15 Mar 2024 07:42:53 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-591842
AcquireTime: <unset>
RenewTime: Fri, 15 Mar 2024 07:51:39 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Fri, 15 Mar 2024 07:46:56 +0000 Fri, 15 Mar 2024 07:42:47 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 15 Mar 2024 07:46:56 +0000 Fri, 15 Mar 2024 07:42:47 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 15 Mar 2024 07:46:56 +0000 Fri, 15 Mar 2024 07:42:47 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 15 Mar 2024 07:46:56 +0000 Fri, 15 Mar 2024 07:43:12 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-591842
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022504Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022504Ki
pods: 110
System Info:
Machine ID: 682867af244d4aed830cdfbe1067ba7b
System UUID: 6b20f7fb-06d7-4232-92df-bac481c756ba
Boot ID: be4a23ea-b3ea-44f1-92fd-06f8e96fb1b3
Kernel Version: 5.15.0-1055-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.6.28
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m35s
kube-system coredns-74ff55c5b-5zhc9 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m31s
kube-system etcd-old-k8s-version-591842 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m38s
kube-system kindnet-9mqv4 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m31s
kube-system kube-apiserver-old-k8s-version-591842 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m38s
kube-system kube-controller-manager-old-k8s-version-591842 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m38s
kube-system kube-proxy-pdn2n 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m31s
kube-system kube-scheduler-old-k8s-version-591842 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m38s
kube-system metrics-server-9975d5f86-9j72g 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m24s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m30s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-2wnk8 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m29s
kubernetes-dashboard kubernetes-dashboard-cd95d586-7fkx2 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m29s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m57s (x5 over 8m57s) kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m57s (x5 over 8m57s) kubelet Node old-k8s-version-591842 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m57s (x5 over 8m57s) kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m57s kubelet Updated Node Allocatable limit across pods
Normal Starting 8m38s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m38s kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m38s kubelet Node old-k8s-version-591842 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m38s kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m38s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m31s kubelet Node old-k8s-version-591842 status is now: NodeReady
Normal Starting 8m29s kube-proxy Starting kube-proxy.
Normal Starting 5m56s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m56s (x8 over 5m56s) kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m56s (x7 over 5m56s) kubelet Node old-k8s-version-591842 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m56s (x8 over 5m56s) kubelet Node old-k8s-version-591842 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m56s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m43s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.001081] FS-Cache: O-key=[8] '10713b0000000000'
[ +0.000757] FS-Cache: N-cookie c=000000d2 [p=000000c9 fl=2 nc=0 na=1]
[ +0.001017] FS-Cache: N-cookie d=000000007332a028{9p.inode} n=00000000c74f53c1
[ +0.001220] FS-Cache: N-key=[8] '10713b0000000000'
[ +0.003939] FS-Cache: Duplicate cookie detected
[ +0.000819] FS-Cache: O-cookie c=000000cc [p=000000c9 fl=226 nc=0 na=1]
[ +0.001156] FS-Cache: O-cookie d=000000007332a028{9p.inode} n=000000005de1f097
[ +0.001196] FS-Cache: O-key=[8] '10713b0000000000'
[ +0.000843] FS-Cache: N-cookie c=000000d3 [p=000000c9 fl=2 nc=0 na=1]
[ +0.001015] FS-Cache: N-cookie d=000000007332a028{9p.inode} n=000000008b64c896
[ +0.001181] FS-Cache: N-key=[8] '10713b0000000000'
[ +3.002798] FS-Cache: Duplicate cookie detected
[ +0.000747] FS-Cache: O-cookie c=000000ca [p=000000c9 fl=226 nc=0 na=1]
[ +0.001129] FS-Cache: O-cookie d=000000007332a028{9p.inode} n=000000007ad487f1
[ +0.001202] FS-Cache: O-key=[8] '0f713b0000000000'
[ +0.000802] FS-Cache: N-cookie c=000000d5 [p=000000c9 fl=2 nc=0 na=1]
[ +0.001168] FS-Cache: N-cookie d=000000007332a028{9p.inode} n=000000005145f6ae
[ +0.001258] FS-Cache: N-key=[8] '0f713b0000000000'
[ +0.293404] FS-Cache: Duplicate cookie detected
[ +0.000810] FS-Cache: O-cookie c=000000cf [p=000000c9 fl=226 nc=0 na=1]
[ +0.001101] FS-Cache: O-cookie d=000000007332a028{9p.inode} n=000000005b55082f
[ +0.001118] FS-Cache: O-key=[8] '15713b0000000000'
[ +0.000729] FS-Cache: N-cookie c=000000d6 [p=000000c9 fl=2 nc=0 na=1]
[ +0.000949] FS-Cache: N-cookie d=000000007332a028{9p.inode} n=00000000c74f53c1
[ +0.001079] FS-Cache: N-key=[8] '15713b0000000000'
==> etcd [288c914996f04026c4b46ed1ae770f4350127a00f6f762dc1f2c827e02e963bf] <==
raft2024/03/15 07:42:47 INFO: ea7e25599daad906 became candidate at term 2
raft2024/03/15 07:42:47 INFO: ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2
raft2024/03/15 07:42:47 INFO: ea7e25599daad906 became leader at term 2
raft2024/03/15 07:42:47 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-03-15 07:42:47.946052 I | etcdserver: setting up the initial cluster version to 3.4
2024-03-15 07:42:47.947148 N | etcdserver/membership: set the initial cluster version to 3.4
2024-03-15 07:42:47.947318 I | etcdserver/api: enabled capabilities for version 3.4
2024-03-15 07:42:47.947410 I | etcdserver: published {Name:old-k8s-version-591842 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-03-15 07:42:47.947612 I | embed: ready to serve client requests
2024-03-15 07:42:47.948379 I | embed: ready to serve client requests
2024-03-15 07:42:47.953339 I | embed: serving client requests on 192.168.76.2:2379
2024-03-15 07:42:47.987902 I | embed: serving client requests on 127.0.0.1:2379
2024-03-15 07:43:11.153334 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:43:20.867847 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:43:30.868030 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:43:40.867855 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:43:50.868056 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:00.868059 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:10.867827 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:20.868141 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:30.868075 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:40.868019 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:44:50.868061 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:45:00.868195 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:45:10.869116 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [3e9f3d24aea76b58fc8b3cc405e7a29d5223e2175c00b4ea4e8698e262cc71fe] <==
2024-03-15 07:47:43.262444 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:47:53.262456 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:03.262521 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:13.262376 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:23.262501 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:33.262762 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:43.262537 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:48:53.262403 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:03.262369 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:13.262460 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:23.262356 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:33.262367 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:43.262409 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:49:53.262541 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:03.262290 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:13.263042 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:23.262355 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:33.262422 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:43.262438 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:50:53.262486 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:51:03.262428 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:51:13.262347 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:51:23.262595 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:51:33.262520 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-15 07:51:43.262523 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
07:51:43 up 16:34, 0 users, load average: 1.93, 1.94, 2.54
Linux old-k8s-version-591842 5.15.0-1055-aws #60~20.04.1-Ubuntu SMP Thu Feb 22 15:54:21 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [7113762b904f49fb25d8457f9978706dc59279f68df6e804674cc618fc850a4e] <==
podIP = 192.168.76.2
I0315 07:43:13.016342 1 main.go:116] setting mtu 1500 for CNI
I0315 07:43:13.016366 1 main.go:146] kindnetd IP family: "ipv4"
I0315 07:43:13.016378 1 main.go:150] noMask IPv4 subnets: [10.244.0.0/16]
I0315 07:43:43.244220 1 main.go:191] Failed to get nodes, retrying after error: Get "https://10.96.0.1:443/api/v1/nodes": dial tcp 10.96.0.1:443: i/o timeout
I0315 07:43:43.308320 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:43:43.308355 1 main.go:227] handling current node
I0315 07:43:53.342015 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:43:53.342225 1 main.go:227] handling current node
I0315 07:44:03.371677 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:03.371708 1 main.go:227] handling current node
I0315 07:44:13.375796 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:13.375822 1 main.go:227] handling current node
I0315 07:44:23.409781 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:23.409981 1 main.go:227] handling current node
I0315 07:44:33.441300 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:33.441331 1 main.go:227] handling current node
I0315 07:44:43.451170 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:43.451200 1 main.go:227] handling current node
I0315 07:44:53.483519 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:44:53.483712 1 main.go:227] handling current node
I0315 07:45:03.528812 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:45:03.528842 1 main.go:227] handling current node
I0315 07:45:13.549196 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:45:13.549393 1 main.go:227] handling current node
==> kindnet [d3767d4efec190990e83e3395e150808bcbe00b98bac2746ed31bbba3417e842] <==
I0315 07:49:39.118817 1 main.go:227] handling current node
I0315 07:49:49.127917 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:49:49.127949 1 main.go:227] handling current node
I0315 07:49:59.136359 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:49:59.136387 1 main.go:227] handling current node
I0315 07:50:09.151701 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:09.151730 1 main.go:227] handling current node
I0315 07:50:19.155498 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:19.155528 1 main.go:227] handling current node
I0315 07:50:29.166180 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:29.166284 1 main.go:227] handling current node
I0315 07:50:39.175413 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:39.175442 1 main.go:227] handling current node
I0315 07:50:49.186432 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:49.186460 1 main.go:227] handling current node
I0315 07:50:59.194189 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:50:59.194219 1 main.go:227] handling current node
I0315 07:51:09.207493 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:51:09.207691 1 main.go:227] handling current node
I0315 07:51:19.223738 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:51:19.223777 1 main.go:227] handling current node
I0315 07:51:29.229264 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:51:29.229490 1 main.go:227] handling current node
I0315 07:51:39.241009 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0315 07:51:39.241140 1 main.go:227] handling current node
==> kube-apiserver [3aee10dec5ed07dafc19f5a933fa5ded3a656c5035a025358b33b9e64d0af465] <==
I0315 07:48:17.528052 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:48:17.528061 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0315 07:48:51.829985 1 client.go:360] parsed scheme: "passthrough"
I0315 07:48:51.830062 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:48:51.830071 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0315 07:48:58.985569 1 handler_proxy.go:102] no RequestInfo found in the context
E0315 07:48:58.985639 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0315 07:48:58.985649 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0315 07:49:24.595911 1 client.go:360] parsed scheme: "passthrough"
I0315 07:49:24.595952 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:49:24.595961 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0315 07:49:57.879448 1 client.go:360] parsed scheme: "passthrough"
I0315 07:49:57.879733 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:49:57.879843 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0315 07:50:41.122869 1 client.go:360] parsed scheme: "passthrough"
I0315 07:50:41.122912 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:50:41.122921 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0315 07:50:57.177921 1 handler_proxy.go:102] no RequestInfo found in the context
E0315 07:50:57.177984 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0315 07:50:57.177994 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0315 07:51:18.743244 1 client.go:360] parsed scheme: "passthrough"
I0315 07:51:18.743291 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:51:18.743301 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [aa813eda9f1da5af9142848c19fa43cf2a6a68098ca63c5783479ec6681835d5] <==
I0315 07:42:54.484830 1 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0315 07:42:54.484860 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0315 07:42:54.500043 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0315 07:42:54.503977 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0315 07:42:54.504001 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0315 07:42:54.932842 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0315 07:42:54.968837 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0315 07:42:55.082404 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I0315 07:42:55.083876 1 controller.go:606] quota admission added evaluator for: endpoints
I0315 07:42:55.097974 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0315 07:42:56.129426 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0315 07:42:56.805673 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0315 07:42:56.853017 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0315 07:43:05.324574 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0315 07:43:12.072205 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0315 07:43:12.136280 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0315 07:43:32.816419 1 client.go:360] parsed scheme: "passthrough"
I0315 07:43:32.816463 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:43:32.816471 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0315 07:44:10.772752 1 client.go:360] parsed scheme: "passthrough"
I0315 07:44:10.772793 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:44:10.772890 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0315 07:44:40.924613 1 client.go:360] parsed scheme: "passthrough"
I0315 07:44:40.924654 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0315 07:44:40.924662 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [39d3647417de7d1e56cd0022025c95051ea33f8094101c1d1e18fb578da8dac2] <==
W0315 07:47:20.553708 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:47:46.606394 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:47:52.204196 1 request.go:655] Throttling request took 1.048280382s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0315 07:47:53.055612 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:48:17.108306 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:48:24.706208 1 request.go:655] Throttling request took 1.048848825s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0315 07:48:25.557699 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:48:47.610182 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:48:57.208218 1 request.go:655] Throttling request took 1.04838658s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0315 07:48:58.059783 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:49:18.112092 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:49:29.710344 1 request.go:655] Throttling request took 1.048332514s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0315 07:49:30.561729 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:49:48.613884 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:50:02.212142 1 request.go:655] Throttling request took 1.047923232s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0315 07:50:03.064166 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:50:19.115614 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:50:34.714836 1 request.go:655] Throttling request took 1.048421865s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0315 07:50:35.566502 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:50:49.617318 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:51:07.216979 1 request.go:655] Throttling request took 1.048037246s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0315 07:51:08.068707 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0315 07:51:20.119737 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0315 07:51:39.720896 1 request.go:655] Throttling request took 1.048288583s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0315 07:51:40.572580 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [6e6c79cda832629dbbd2f62624a876664c7a5c6d3be1a6dda27e404d9b60545d] <==
I0315 07:43:12.140154 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-legacy-unknown
I0315 07:43:12.140181 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-kubelet-serving
I0315 07:43:12.140420 1 shared_informer.go:247] Caches are synced for certificate-csrsigning-kubelet-client
I0315 07:43:12.150081 1 shared_informer.go:247] Caches are synced for certificate-csrapproving
I0315 07:43:12.156725 1 shared_informer.go:240] Waiting for caches to sync for resource quota
I0315 07:43:12.185070 1 event.go:291] "Event occurred" object="kube-system/kube-apiserver-old-k8s-version-591842" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0315 07:43:12.187626 1 shared_informer.go:247] Caches are synced for ClusterRoleAggregator
I0315 07:43:12.213019 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-9mqv4"
I0315 07:43:12.237945 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-pdn2n"
I0315 07:43:12.318945 1 shared_informer.go:247] Caches are synced for resource quota
I0315 07:43:12.319015 1 shared_informer.go:247] Caches are synced for endpoint_slice
I0315 07:43:12.359630 1 shared_informer.go:247] Caches are synced for resource quota
I0315 07:43:12.362641 1 shared_informer.go:247] Caches are synced for endpoint_slice_mirroring
E0315 07:43:12.420673 1 daemon_controller.go:320] kube-system/kube-proxy failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-proxy", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"007fb872-eba2-4195-bdfd-80d8e1a0764b", ResourceVersion:"279", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63846085376, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubeadm", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001937ac0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001937ae0)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.
LabelSelector)(0x4001937b00), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"kube-proxy", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Gl
usterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0x4001800280), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001937
b20), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeS
ource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001937b40), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil),
AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kube-proxy", Image:"k8s.gcr.io/kube-proxy:v1.20.0", Command:[]string{"/usr/local/bin/kube-proxy", "--config=/var/lib/kube-proxy/config.conf", "--hostname-override=$(NODE_NAME)"}, Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001937b80)}}, Resources:v1.R
esourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList(nil)}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"kube-proxy", ReadOnly:false, MountPath:"/var/lib/kube-proxy", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x40017bcb40), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPo
licy:"Always", TerminationGracePeriodSeconds:(*int64)(0x40016b1e38), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"kube-proxy", DeprecatedServiceAccount:"kube-proxy", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x4000a7a380), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), Runtime
ClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x40008efa20)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x40016b1e88)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again
I0315 07:43:12.453737 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0315 07:43:12.733669 1 shared_informer.go:247] Caches are synced for garbage collector
I0315 07:43:12.733695 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0315 07:43:12.753958 1 shared_informer.go:247] Caches are synced for garbage collector
I0315 07:43:13.270380 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0315 07:43:13.294407 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-jxqhl"
I0315 07:43:17.103365 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0315 07:45:18.467637 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
I0315 07:45:18.546005 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"metrics-server-9975d5f86-\" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount \"metrics-server\" not found"
E0315 07:45:18.600297 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with pods "metrics-server-9975d5f86-" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount "metrics-server" not found
E0315 07:45:18.683249 1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [1909102cb4c316a85c844de34fe80392e27c7c27b923245563422bfbd4f97439] <==
I0315 07:43:14.300222 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0315 07:43:14.300317 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0315 07:43:14.319706 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0315 07:43:14.319817 1 server_others.go:185] Using iptables Proxier.
I0315 07:43:14.320026 1 server.go:650] Version: v1.20.0
I0315 07:43:14.320658 1 config.go:315] Starting service config controller
I0315 07:43:14.320722 1 shared_informer.go:240] Waiting for caches to sync for service config
I0315 07:43:14.320758 1 config.go:224] Starting endpoint slice config controller
I0315 07:43:14.323755 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0315 07:43:14.423179 1 shared_informer.go:247] Caches are synced for service config
I0315 07:43:14.423953 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [24cc1f1e367cbeae35948802bbf43f632ee4bd55c039a00d8c5a76566109f2c3] <==
I0315 07:46:00.660048 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0315 07:46:00.661929 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0315 07:46:00.683993 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0315 07:46:00.684239 1 server_others.go:185] Using iptables Proxier.
I0315 07:46:00.684624 1 server.go:650] Version: v1.20.0
I0315 07:46:00.685459 1 config.go:315] Starting service config controller
I0315 07:46:00.685485 1 shared_informer.go:240] Waiting for caches to sync for service config
I0315 07:46:00.685505 1 config.go:224] Starting endpoint slice config controller
I0315 07:46:00.685628 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0315 07:46:00.785608 1 shared_informer.go:247] Caches are synced for service config
I0315 07:46:00.785809 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [19906a36a5bc65fa9053fa523d8e7bd5e048a857bef608d62d4b59e39df92423] <==
I0315 07:45:51.336575 1 serving.go:331] Generated self-signed cert in-memory
W0315 07:45:56.153108 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0315 07:45:56.153150 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0315 07:45:56.153206 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0315 07:45:56.153212 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0315 07:45:56.408405 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0315 07:45:56.408923 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0315 07:45:56.408932 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0315 07:45:56.408947 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0315 07:45:56.510913 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [eb46f252b98e775a9323a10c057054ca3ab5d7e9f5f3fc457cb1cc0b0674781d] <==
W0315 07:42:53.617683 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0315 07:42:53.617717 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0315 07:42:53.617731 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0315 07:42:53.617737 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0315 07:42:53.725308 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0315 07:42:53.725663 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0315 07:42:53.727737 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0315 07:42:53.728388 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0315 07:42:53.757539 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0315 07:42:53.757870 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0315 07:42:53.758098 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0315 07:42:53.758294 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0315 07:42:53.758526 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0315 07:42:53.766252 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0315 07:42:53.768309 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0315 07:42:53.769726 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0315 07:42:53.770258 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0315 07:42:53.771037 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0315 07:42:53.771319 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0315 07:42:53.774165 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0315 07:42:54.660551 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0315 07:42:54.729354 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0315 07:42:54.776833 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0315 07:42:54.778625 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
I0315 07:42:55.228339 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: I0315 07:50:17.644466 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.645285 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:17 old-k8s-version-591842 kubelet[663]: E0315 07:50:17.647267 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: I0315 07:50:28.643990 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:50:28 old-k8s-version-591842 kubelet[663]: E0315 07:50:28.644332 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:29 old-k8s-version-591842 kubelet[663]: E0315 07:50:29.645326 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:50:42 old-k8s-version-591842 kubelet[663]: E0315 07:50:42.644973 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: I0315 07:50:43.644084 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:50:43 old-k8s-version-591842 kubelet[663]: E0315 07:50:43.644572 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.644836 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: I0315 07:50:54.644988 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:50:54 old-k8s-version-591842 kubelet[663]: E0315 07:50:54.645785 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: I0315 07:51:07.644862 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.645648 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:07 old-k8s-version-591842 kubelet[663]: E0315 07:51:07.651347 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:19 old-k8s-version-591842 kubelet[663]: E0315 07:51:19.648408 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: I0315 07:51:21.644069 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:51:21 old-k8s-version-591842 kubelet[663]: E0315 07:51:21.644468 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:32 old-k8s-version-591842 kubelet[663]: E0315 07:51:32.647274 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 15 07:51:36 old-k8s-version-591842 kubelet[663]: I0315 07:51:36.644025 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: c8441ee92457c869613120768699a33ceb84581653db7649198ccb9e71d8c473
Mar 15 07:51:36 old-k8s-version-591842 kubelet[663]: E0315 07:51:36.644366 663 pod_workers.go:191] Error syncing pod 4f033064-ca42-40d1-8805-5d3ff76c1fa1 ("dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-2wnk8_kubernetes-dashboard(4f033064-ca42-40d1-8805-5d3ff76c1fa1)"
Mar 15 07:51:43 old-k8s-version-591842 kubelet[663]: E0315 07:51:43.662949 663 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Mar 15 07:51:43 old-k8s-version-591842 kubelet[663]: E0315 07:51:43.663022 663 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Mar 15 07:51:43 old-k8s-version-591842 kubelet[663]: E0315 07:51:43.663253 663 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-h2gnq,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-9j72g_kube-system(84487c4
f-2bb4-4d4f-9257-aaf715457d3f): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Mar 15 07:51:43 old-k8s-version-591842 kubelet[663]: E0315 07:51:43.663300 663 pod_workers.go:191] Error syncing pod 84487c4f-2bb4-4d4f-9257-aaf715457d3f ("metrics-server-9975d5f86-9j72g_kube-system(84487c4f-2bb4-4d4f-9257-aaf715457d3f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
==> kubernetes-dashboard [141c1019fad2f75625f7533bb7db48335b3215e4c610540fdc2307cccb03f95c] <==
2024/03/15 07:46:23 Using namespace: kubernetes-dashboard
2024/03/15 07:46:23 Using in-cluster config to connect to apiserver
2024/03/15 07:46:23 Using secret token for csrf signing
2024/03/15 07:46:23 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/03/15 07:46:23 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/03/15 07:46:23 Successful initial request to the apiserver, version: v1.20.0
2024/03/15 07:46:23 Generating JWE encryption key
2024/03/15 07:46:23 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/03/15 07:46:23 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/03/15 07:46:23 Initializing JWE encryption key from synchronized object
2024/03/15 07:46:23 Creating in-cluster Sidecar client
2024/03/15 07:46:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:46:23 Serving insecurely on HTTP port: 9090
2024/03/15 07:46:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:47:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:47:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:48:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:48:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:49:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:49:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:50:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:50:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:51:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/15 07:46:23 Starting overwatch
==> storage-provisioner [4518d148ada79ac861923312551230f9f3718d390d5b3600bb80715fde6119d2] <==
I0315 07:46:00.673678 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0315 07:46:30.676524 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [ae03a9797792da6de45dd59f24e27789b8b12426efd48a8a33efb4524e861803] <==
I0315 07:46:41.767740 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0315 07:46:41.787987 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0315 07:46:41.788041 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0315 07:46:59.261313 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0315 07:46:59.261642 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-591842_087faa41-69b4-493d-a540-51db6d805520!
I0315 07:46:59.262534 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"15d8587f-a922-4470-844a-e6979cddac33", APIVersion:"v1", ResourceVersion:"879", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-591842_087faa41-69b4-493d-a540-51db6d805520 became leader
I0315 07:46:59.362183 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-591842_087faa41-69b4-493d-a540-51db6d805520!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-591842 -n old-k8s-version-591842
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-591842 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-9j72g
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-591842 describe pod metrics-server-9975d5f86-9j72g
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-591842 describe pod metrics-server-9975d5f86-9j72g: exit status 1 (163.379105ms)
** stderr **
E0315 07:51:45.573544 3503809 memcache.go:287] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0315 07:51:45.578712 3503809 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0315 07:51:45.580952 3503809 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0315 07:51:45.584211 3503809 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0315 07:51:45.593347 3503809 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0315 07:51:45.608210 3503809 memcache.go:121] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
Error from server (NotFound): pods "metrics-server-9975d5f86-9j72g" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-591842 describe pod metrics-server-9975d5f86-9j72g: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (373.89s)