=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-014065 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0
E0416 00:42:50.201499 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/custom-flannel-128493/client.crt: no such file or directory
E0416 00:42:57.645341 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.650579 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.660820 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.681068 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.721309 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.801706 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:57.962423 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:58.283478 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:42:58.924112 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:00.204514 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:02.765103 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:03.303573 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/enable-default-cni-128493/client.crt: no such file or directory
E0416 00:43:03.607458 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kindnet-128493/client.crt: no such file or directory
E0416 00:43:07.886133 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:18.126951 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:31.298240 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kindnet-128493/client.crt: no such file or directory
E0416 00:43:38.607505 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:43:51.585565 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/false-128493/client.crt: no such file or directory
E0416 00:43:57.942883 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:57.948326 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:57.958595 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:57.978864 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:58.019185 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:58.099455 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:58.259945 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:58.580609 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:43:58.957315 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/calico-128493/client.crt: no such file or directory
E0416 00:43:59.220757 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:00.514738 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:03.078131 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:08.199122 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:18.439338 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:19.567754 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:44:25.224513 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/enable-default-cni-128493/client.crt: no such file or directory
E0416 00:44:26.643357 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/calico-128493/client.crt: no such file or directory
E0416 00:44:38.919987 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:44:48.451838 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/skaffold-421454/client.crt: no such file or directory
E0416 00:45:05.392504 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.397765 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.408019 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.428392 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.468726 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.549017 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:05.709392 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:06.029961 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:06.358021 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/custom-flannel-128493/client.crt: no such file or directory
E0416 00:45:06.670620 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:07.951512 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:10.512575 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:15.633059 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:18.834498 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/addons-716538/client.crt: no such file or directory
E0416 00:45:19.880406 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/bridge-128493/client.crt: no such file or directory
E0416 00:45:25.873620 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:45:34.041729 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/custom-flannel-128493/client.crt: no such file or directory
E0416 00:45:41.488343 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/flannel-128493/client.crt: no such file or directory
E0416 00:45:46.353844 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/kubenet-128493/client.crt: no such file or directory
E0416 00:46:07.742965 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/false-128493/client.crt: no such file or directory
E0416 00:46:20.762198 7563 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/auto-128493/client.crt: no such file or directory
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-014065 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0: exit status 102 (6m8.655319296s)
-- stdout --
* [old-k8s-version-014065] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18647
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18647-2210/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18647-2210/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
* Using the docker driver based on existing profile
* Starting "old-k8s-version-014065" primary control-plane node in "old-k8s-version-014065" cluster
* Pulling base image v0.0.43-1713215244-18647 ...
* Restarting existing docker container for "old-k8s-version-014065" ...
* Preparing Kubernetes v1.20.0 on Docker 26.0.1 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-014065 addons enable metrics-server
* Enabled addons: metrics-server, default-storageclass, storage-provisioner, dashboard
-- /stdout --
** stderr **
I0416 00:42:40.650687 369017 out.go:291] Setting OutFile to fd 1 ...
I0416 00:42:40.650860 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:42:40.650892 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:42:40.650912 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:42:40.651179 369017 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18647-2210/.minikube/bin
I0416 00:42:40.651617 369017 out.go:298] Setting JSON to false
I0416 00:42:40.652782 369017 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":5096,"bootTime":1713223065,"procs":273,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1057-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0416 00:42:40.652882 369017 start.go:139] virtualization:
I0416 00:42:40.657149 369017 out.go:177] * [old-k8s-version-014065] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0416 00:42:40.659310 369017 out.go:177] - MINIKUBE_LOCATION=18647
I0416 00:42:40.659393 369017 notify.go:220] Checking for updates...
I0416 00:42:40.664118 369017 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0416 00:42:40.666095 369017 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:42:40.668274 369017 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18647-2210/.minikube
I0416 00:42:40.670450 369017 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0416 00:42:40.672695 369017 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0416 00:42:40.675191 369017 config.go:182] Loaded profile config "old-k8s-version-014065": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0416 00:42:40.678294 369017 out.go:177] * Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
I0416 00:42:40.680735 369017 driver.go:392] Setting default libvirt URI to qemu:///system
I0416 00:42:40.704616 369017 docker.go:122] docker version: linux-26.0.1:Docker Engine - Community
I0416 00:42:40.704740 369017 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0416 00:42:40.784256 369017 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:52 SystemTime:2024-04-16 00:42:40.771569696 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1057-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.26.1]] Warnings:<nil>}}
I0416 00:42:40.784375 369017 docker.go:295] overlay module found
I0416 00:42:40.787819 369017 out.go:177] * Using the docker driver based on existing profile
I0416 00:42:40.789398 369017 start.go:297] selected driver: docker
I0416 00:42:40.789418 369017 start.go:901] validating driver "docker" against &{Name:old-k8s-version-014065 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-014065 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountSt
ring:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:42:40.789550 369017 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0416 00:42:40.790224 369017 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0416 00:42:40.858742 369017 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:32 OomKillDisable:true NGoroutines:52 SystemTime:2024-04-16 00:42:40.838986442 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1057-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.26.1]] Warnings:<nil>}}
I0416 00:42:40.859116 369017 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0416 00:42:40.859187 369017 cni.go:84] Creating CNI manager for ""
I0416 00:42:40.859307 369017 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I0416 00:42:40.859362 369017 start.go:340] cluster config:
{Name:old-k8s-version-014065 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-014065 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountI
P: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:42:40.862959 369017 out.go:177] * Starting "old-k8s-version-014065" primary control-plane node in "old-k8s-version-014065" cluster
I0416 00:42:40.864551 369017 cache.go:121] Beginning downloading kic base image for docker with docker
I0416 00:42:40.866528 369017 out.go:177] * Pulling base image v0.0.43-1713215244-18647 ...
I0416 00:42:40.868467 369017 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker
I0416 00:42:40.868519 369017 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18647-2210/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4
I0416 00:42:40.868539 369017 cache.go:56] Caching tarball of preloaded images
I0416 00:42:40.868571 369017 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af in local docker daemon
I0416 00:42:40.868617 369017 preload.go:173] Found /home/jenkins/minikube-integration/18647-2210/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4 in cache, skipping download
I0416 00:42:40.868628 369017 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on docker
I0416 00:42:40.868740 369017 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/config.json ...
I0416 00:42:40.883252 369017 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af in local docker daemon, skipping pull
I0416 00:42:40.883280 369017 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af exists in daemon, skipping load
I0416 00:42:40.883301 369017 cache.go:194] Successfully downloaded all kic artifacts
I0416 00:42:40.883337 369017 start.go:360] acquireMachinesLock for old-k8s-version-014065: {Name:mk76975395468af310769b039e692fcb88b5961e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0416 00:42:40.883407 369017 start.go:364] duration metric: took 43.027µs to acquireMachinesLock for "old-k8s-version-014065"
I0416 00:42:40.883433 369017 start.go:96] Skipping create...Using existing machine configuration
I0416 00:42:40.883442 369017 fix.go:54] fixHost starting:
I0416 00:42:40.883714 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:40.900618 369017 fix.go:112] recreateIfNeeded on old-k8s-version-014065: state=Stopped err=<nil>
W0416 00:42:40.900660 369017 fix.go:138] unexpected machine state, will restart: <nil>
I0416 00:42:40.903302 369017 out.go:177] * Restarting existing docker container for "old-k8s-version-014065" ...
I0416 00:42:40.905375 369017 cli_runner.go:164] Run: docker start old-k8s-version-014065
I0416 00:42:41.230339 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:41.254892 369017 kic.go:430] container "old-k8s-version-014065" state is running.
I0416 00:42:41.255408 369017 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-014065
I0416 00:42:41.277517 369017 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/config.json ...
I0416 00:42:41.277898 369017 machine.go:94] provisionDockerMachine start ...
I0416 00:42:41.278073 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:41.297808 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:41.298129 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:41.298140 369017 main.go:141] libmachine: About to run SSH command:
hostname
I0416 00:42:41.298884 369017 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0416 00:42:44.446753 369017 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-014065
I0416 00:42:44.446777 369017 ubuntu.go:169] provisioning hostname "old-k8s-version-014065"
I0416 00:42:44.446839 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:44.462291 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:44.462542 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:44.462561 369017 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-014065 && echo "old-k8s-version-014065" | sudo tee /etc/hostname
I0416 00:42:44.619722 369017 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-014065
I0416 00:42:44.619809 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:44.637431 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:44.637691 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:44.637712 369017 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-014065' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-014065/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-014065' | sudo tee -a /etc/hosts;
fi
fi
I0416 00:42:44.783431 369017 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:42:44.783523 369017 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18647-2210/.minikube CaCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18647-2210/.minikube}
I0416 00:42:44.783572 369017 ubuntu.go:177] setting up certificates
I0416 00:42:44.783603 369017 provision.go:84] configureAuth start
I0416 00:42:44.783695 369017 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-014065
I0416 00:42:44.799622 369017 provision.go:143] copyHostCerts
I0416 00:42:44.799700 369017 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem, removing ...
I0416 00:42:44.799715 369017 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem
I0416 00:42:44.799795 369017 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem (1078 bytes)
I0416 00:42:44.799954 369017 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem, removing ...
I0416 00:42:44.799967 369017 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem
I0416 00:42:44.799997 369017 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem (1123 bytes)
I0416 00:42:44.800058 369017 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem, removing ...
I0416 00:42:44.800067 369017 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem
I0416 00:42:44.800093 369017 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem (1679 bytes)
I0416 00:42:44.800143 369017 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-014065 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-014065]
I0416 00:42:45.012209 369017 provision.go:177] copyRemoteCerts
I0416 00:42:45.012301 369017 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0416 00:42:45.012349 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:45.062155 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:45.200098 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0416 00:42:45.258251 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0416 00:42:45.340053 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0416 00:42:45.389453 369017 provision.go:87] duration metric: took 605.820474ms to configureAuth
I0416 00:42:45.389480 369017 ubuntu.go:193] setting minikube options for container-runtime
I0416 00:42:45.389708 369017 config.go:182] Loaded profile config "old-k8s-version-014065": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0416 00:42:45.389862 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:45.419055 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:45.419344 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:45.419355 369017 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0416 00:42:45.576308 369017 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0416 00:42:45.576331 369017 ubuntu.go:71] root file system type: overlay
I0416 00:42:45.576452 369017 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0416 00:42:45.576526 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:45.593791 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:45.594064 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:45.594148 369017 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0416 00:42:45.755944 369017 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0416 00:42:45.756059 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:45.774814 369017 main.go:141] libmachine: Using SSH client type: native
I0416 00:42:45.775072 369017 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33137 <nil> <nil>}
I0416 00:42:45.775096 369017 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0416 00:42:45.929028 369017 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:42:45.929054 369017 machine.go:97] duration metric: took 4.651142154s to provisionDockerMachine
I0416 00:42:45.929067 369017 start.go:293] postStartSetup for "old-k8s-version-014065" (driver="docker")
I0416 00:42:45.929078 369017 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0416 00:42:45.929159 369017 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0416 00:42:45.929201 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:45.949654 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:46.053819 369017 ssh_runner.go:195] Run: cat /etc/os-release
I0416 00:42:46.057318 369017 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0416 00:42:46.057362 369017 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0416 00:42:46.057376 369017 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0416 00:42:46.057382 369017 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0416 00:42:46.057392 369017 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-2210/.minikube/addons for local assets ...
I0416 00:42:46.057454 369017 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-2210/.minikube/files for local assets ...
I0416 00:42:46.057528 369017 filesync.go:149] local asset: /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem -> 75632.pem in /etc/ssl/certs
I0416 00:42:46.057647 369017 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0416 00:42:46.067853 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem --> /etc/ssl/certs/75632.pem (1708 bytes)
I0416 00:42:46.095193 369017 start.go:296] duration metric: took 166.111499ms for postStartSetup
I0416 00:42:46.095362 369017 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0416 00:42:46.095428 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:46.110487 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:46.212470 369017 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0416 00:42:46.217233 369017 fix.go:56] duration metric: took 5.333784607s for fixHost
I0416 00:42:46.217256 369017 start.go:83] releasing machines lock for "old-k8s-version-014065", held for 5.333835855s
I0416 00:42:46.217339 369017 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-014065
I0416 00:42:46.235046 369017 ssh_runner.go:195] Run: cat /version.json
I0416 00:42:46.235139 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:46.235423 369017 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0416 00:42:46.235471 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:46.252418 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:46.260658 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:46.350842 369017 ssh_runner.go:195] Run: systemctl --version
I0416 00:42:46.478569 369017 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0416 00:42:46.483752 369017 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0416 00:42:46.505023 369017 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0416 00:42:46.505119 369017 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0416 00:42:46.523572 369017 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0416 00:42:46.541763 369017 cni.go:308] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0416 00:42:46.541793 369017 start.go:494] detecting cgroup driver to use...
I0416 00:42:46.541827 369017 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0416 00:42:46.541929 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0416 00:42:46.559907 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0416 00:42:46.570407 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0416 00:42:46.581203 369017 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0416 00:42:46.581266 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0416 00:42:46.597430 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0416 00:42:46.609876 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0416 00:42:46.623569 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0416 00:42:46.639861 369017 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0416 00:42:46.650339 369017 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0416 00:42:46.660506 369017 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0416 00:42:46.669239 369017 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0416 00:42:46.687918 369017 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:42:46.773278 369017 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0416 00:42:46.878574 369017 start.go:494] detecting cgroup driver to use...
I0416 00:42:46.878672 369017 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0416 00:42:46.878756 369017 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0416 00:42:46.892579 369017 cruntime.go:279] skipping containerd shutdown because we are bound to it
I0416 00:42:46.892712 369017 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0416 00:42:46.906960 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0416 00:42:46.946120 369017 ssh_runner.go:195] Run: which cri-dockerd
I0416 00:42:46.950034 369017 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0416 00:42:46.960271 369017 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0416 00:42:46.982839 369017 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0416 00:42:47.085110 369017 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0416 00:42:47.196394 369017 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0416 00:42:47.196595 369017 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0416 00:42:47.218519 369017 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:42:47.333055 369017 ssh_runner.go:195] Run: sudo systemctl restart docker
I0416 00:42:47.743980 369017 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0416 00:42:47.766270 369017 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0416 00:42:47.791635 369017 out.go:204] * Preparing Kubernetes v1.20.0 on Docker 26.0.1 ...
I0416 00:42:47.791755 369017 cli_runner.go:164] Run: docker network inspect old-k8s-version-014065 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0416 00:42:47.814873 369017 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0416 00:42:47.818679 369017 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:42:47.829749 369017 kubeadm.go:877] updating cluster {Name:old-k8s-version-014065 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-014065 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkin
s:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0416 00:42:47.829883 369017 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker
I0416 00:42:47.829942 369017 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0416 00:42:47.846608 369017 docker.go:685] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.0
registry.k8s.io/kube-proxy:v1.20.0
k8s.gcr.io/kube-apiserver:v1.20.0
registry.k8s.io/kube-apiserver:v1.20.0
k8s.gcr.io/kube-controller-manager:v1.20.0
registry.k8s.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-scheduler:v1.20.0
registry.k8s.io/kube-scheduler:v1.20.0
registry.k8s.io/etcd:3.4.13-0
k8s.gcr.io/etcd:3.4.13-0
registry.k8s.io/coredns:1.7.0
k8s.gcr.io/coredns:1.7.0
registry.k8s.io/pause:3.2
k8s.gcr.io/pause:3.2
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0416 00:42:47.846642 369017 docker.go:615] Images already preloaded, skipping extraction
I0416 00:42:47.846711 369017 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0416 00:42:47.864960 369017 docker.go:685] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.0
registry.k8s.io/kube-proxy:v1.20.0
k8s.gcr.io/kube-apiserver:v1.20.0
registry.k8s.io/kube-apiserver:v1.20.0
k8s.gcr.io/kube-controller-manager:v1.20.0
registry.k8s.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-scheduler:v1.20.0
registry.k8s.io/kube-scheduler:v1.20.0
k8s.gcr.io/etcd:3.4.13-0
registry.k8s.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
registry.k8s.io/coredns:1.7.0
k8s.gcr.io/pause:3.2
registry.k8s.io/pause:3.2
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0416 00:42:47.864983 369017 cache_images.go:84] Images are preloaded, skipping loading
I0416 00:42:47.864994 369017 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.20.0 docker true true} ...
I0416 00:42:47.865112 369017 kubeadm.go:940] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-014065 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-014065 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0416 00:42:47.865188 369017 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0416 00:42:47.915114 369017 cni.go:84] Creating CNI manager for ""
I0416 00:42:47.915146 369017 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I0416 00:42:47.915155 369017 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0416 00:42:47.915284 369017 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-014065 NodeName:old-k8s-version-014065 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0416 00:42:47.915465 369017 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-014065"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0416 00:42:47.915539 369017 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0416 00:42:47.924590 369017 binaries.go:44] Found k8s binaries, skipping transfer
I0416 00:42:47.924664 369017 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0416 00:42:47.933468 369017 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0416 00:42:47.956820 369017 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0416 00:42:47.976296 369017 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2118 bytes)
I0416 00:42:47.996472 369017 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0416 00:42:48.000377 369017 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:42:48.019281 369017 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:42:48.118866 369017 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0416 00:42:48.134694 369017 certs.go:68] Setting up /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065 for IP: 192.168.76.2
I0416 00:42:48.134719 369017 certs.go:194] generating shared ca certs ...
I0416 00:42:48.134736 369017 certs.go:226] acquiring lock for ca certs: {Name:mk0f2c276f9ccc821c50906b5561fa26a27a6ffe Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:42:48.134898 369017 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18647-2210/.minikube/ca.key
I0416 00:42:48.134947 369017 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.key
I0416 00:42:48.134959 369017 certs.go:256] generating profile certs ...
I0416 00:42:48.135056 369017 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/client.key
I0416 00:42:48.135128 369017 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/apiserver.key.04662e63
I0416 00:42:48.135179 369017 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/proxy-client.key
I0416 00:42:48.135393 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563.pem (1338 bytes)
W0416 00:42:48.135436 369017 certs.go:480] ignoring /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563_empty.pem, impossibly tiny 0 bytes
I0416 00:42:48.135449 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem (1679 bytes)
I0416 00:42:48.135475 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem (1078 bytes)
I0416 00:42:48.135503 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem (1123 bytes)
I0416 00:42:48.135537 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem (1679 bytes)
I0416 00:42:48.135589 369017 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem (1708 bytes)
I0416 00:42:48.136255 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0416 00:42:48.175106 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0416 00:42:48.203877 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0416 00:42:48.230738 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0416 00:42:48.256640 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0416 00:42:48.282240 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0416 00:42:48.334891 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0416 00:42:48.380151 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/old-k8s-version-014065/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0416 00:42:48.425926 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0416 00:42:48.459841 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563.pem --> /usr/share/ca-certificates/7563.pem (1338 bytes)
I0416 00:42:48.494467 369017 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem --> /usr/share/ca-certificates/75632.pem (1708 bytes)
I0416 00:42:48.533025 369017 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0416 00:42:48.554900 369017 ssh_runner.go:195] Run: openssl version
I0416 00:42:48.561486 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/75632.pem && ln -fs /usr/share/ca-certificates/75632.pem /etc/ssl/certs/75632.pem"
I0416 00:42:48.580503 369017 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/75632.pem
I0416 00:42:48.585164 369017 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Apr 15 23:43 /usr/share/ca-certificates/75632.pem
I0416 00:42:48.585281 369017 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/75632.pem
I0416 00:42:48.593724 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/75632.pem /etc/ssl/certs/3ec20f2e.0"
I0416 00:42:48.606639 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0416 00:42:48.616860 369017 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0416 00:42:48.620911 369017 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Apr 15 23:38 /usr/share/ca-certificates/minikubeCA.pem
I0416 00:42:48.621026 369017 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0416 00:42:48.629276 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0416 00:42:48.638872 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/7563.pem && ln -fs /usr/share/ca-certificates/7563.pem /etc/ssl/certs/7563.pem"
I0416 00:42:48.649249 369017 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/7563.pem
I0416 00:42:48.652739 369017 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Apr 15 23:43 /usr/share/ca-certificates/7563.pem
I0416 00:42:48.652854 369017 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/7563.pem
I0416 00:42:48.659891 369017 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/7563.pem /etc/ssl/certs/51391683.0"
I0416 00:42:48.672364 369017 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0416 00:42:48.676371 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0416 00:42:48.684325 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0416 00:42:48.691452 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0416 00:42:48.698366 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0416 00:42:48.705601 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0416 00:42:48.713057 369017 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0416 00:42:48.720399 369017 kubeadm.go:391] StartCluster: {Name:old-k8s-version-014065 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-014065 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/
minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:42:48.720577 369017 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0416 00:42:48.736357 369017 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0416 00:42:48.745966 369017 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0416 00:42:48.746065 369017 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0416 00:42:48.746086 369017 kubeadm.go:587] restartPrimaryControlPlane start ...
I0416 00:42:48.746155 369017 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0416 00:42:48.755569 369017 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0416 00:42:48.756197 369017 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-014065" does not appear in /home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:42:48.756459 369017 kubeconfig.go:62] /home/jenkins/minikube-integration/18647-2210/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-014065" cluster setting kubeconfig missing "old-k8s-version-014065" context setting]
I0416 00:42:48.756939 369017 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-2210/kubeconfig: {Name:mk2a4b2f2d98970b43b7e481fd26cc76bda92838 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:42:48.758275 369017 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0416 00:42:48.768640 369017 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.76.2
I0416 00:42:48.768740 369017 kubeadm.go:591] duration metric: took 22.631028ms to restartPrimaryControlPlane
I0416 00:42:48.768758 369017 kubeadm.go:393] duration metric: took 48.375226ms to StartCluster
I0416 00:42:48.768775 369017 settings.go:142] acquiring lock: {Name:mkad41a04993d6fe82f2e16230c6052d1c68b809 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:42:48.768875 369017 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:42:48.769900 369017 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-2210/kubeconfig: {Name:mk2a4b2f2d98970b43b7e481fd26cc76bda92838 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:42:48.770202 369017 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0416 00:42:48.773565 369017 out.go:177] * Verifying Kubernetes components...
I0416 00:42:48.770442 369017 config.go:182] Loaded profile config "old-k8s-version-014065": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0416 00:42:48.770456 369017 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0416 00:42:48.775486 369017 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-014065"
I0416 00:42:48.775506 369017 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:42:48.775517 369017 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-014065"
W0416 00:42:48.775524 369017 addons.go:243] addon storage-provisioner should already be in state true
I0416 00:42:48.775564 369017 host.go:66] Checking if "old-k8s-version-014065" exists ...
I0416 00:42:48.775599 369017 addons.go:69] Setting dashboard=true in profile "old-k8s-version-014065"
I0416 00:42:48.775624 369017 addons.go:234] Setting addon dashboard=true in "old-k8s-version-014065"
W0416 00:42:48.775630 369017 addons.go:243] addon dashboard should already be in state true
I0416 00:42:48.775663 369017 host.go:66] Checking if "old-k8s-version-014065" exists ...
I0416 00:42:48.776009 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:48.776084 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:48.777381 369017 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-014065"
I0416 00:42:48.777431 369017 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-014065"
I0416 00:42:48.777754 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:48.778044 369017 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-014065"
I0416 00:42:48.778077 369017 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-014065"
W0416 00:42:48.778085 369017 addons.go:243] addon metrics-server should already be in state true
I0416 00:42:48.778113 369017 host.go:66] Checking if "old-k8s-version-014065" exists ...
I0416 00:42:48.778490 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:48.833729 369017 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0416 00:42:48.836206 369017 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0416 00:42:48.837388 369017 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:42:48.841447 369017 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0416 00:42:48.839371 369017 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0416 00:42:48.839393 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0416 00:42:48.840475 369017 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-014065"
W0416 00:42:48.849091 369017 addons.go:243] addon default-storageclass should already be in state true
I0416 00:42:48.849127 369017 host.go:66] Checking if "old-k8s-version-014065" exists ...
I0416 00:42:48.849146 369017 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0416 00:42:48.849165 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0416 00:42:48.849230 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:48.849565 369017 cli_runner.go:164] Run: docker container inspect old-k8s-version-014065 --format={{.State.Status}}
I0416 00:42:48.852331 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0416 00:42:48.852359 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0416 00:42:48.850483 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:48.852426 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:48.893378 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:48.903404 369017 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0416 00:42:48.903423 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0416 00:42:48.903510 369017 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-014065
I0416 00:42:48.907517 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:48.912527 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:48.940743 369017 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33137 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/old-k8s-version-014065/id_rsa Username:docker}
I0416 00:42:48.967087 369017 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0416 00:42:48.994875 369017 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-014065" to be "Ready" ...
I0416 00:42:49.037169 369017 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0416 00:42:49.037191 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0416 00:42:49.056713 369017 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0416 00:42:49.056784 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0416 00:42:49.090608 369017 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:42:49.090680 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0416 00:42:49.099051 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0416 00:42:49.112879 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:42:49.114303 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0416 00:42:49.114363 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0416 00:42:49.158880 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0416 00:42:49.158952 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0416 00:42:49.166292 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:42:49.223658 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0416 00:42:49.223730 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0416 00:42:49.292760 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0416 00:42:49.292836 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0416 00:42:49.312936 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.313015 369017 retry.go:31] will retry after 276.652953ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0416 00:42:49.312884 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.313115 369017 retry.go:31] will retry after 253.608697ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.323674 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0416 00:42:49.323748 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0416 00:42:49.349695 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0416 00:42:49.349770 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0416 00:42:49.363130 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.363223 369017 retry.go:31] will retry after 220.114105ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.371991 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0416 00:42:49.372015 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0416 00:42:49.391025 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0416 00:42:49.391048 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0416 00:42:49.409859 369017 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0416 00:42:49.409884 369017 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0416 00:42:49.430528 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0416 00:42:49.508649 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.508693 369017 retry.go:31] will retry after 319.576621ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.567865 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0416 00:42:49.584337 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:42:49.589864 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0416 00:42:49.717337 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.717389 369017 retry.go:31] will retry after 342.735867ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0416 00:42:49.737636 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.737755 369017 retry.go:31] will retry after 480.62207ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0416 00:42:49.737761 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.737828 369017 retry.go:31] will retry after 206.909322ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.828801 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0416 00:42:49.900438 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.900493 369017 retry.go:31] will retry after 504.091881ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:49.945634 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0416 00:42:50.033975 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.034012 369017 retry.go:31] will retry after 428.956014ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.061210 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0416 00:42:50.137117 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.137148 369017 retry.go:31] will retry after 403.493122ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.219467 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0416 00:42:50.291358 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.291395 369017 retry.go:31] will retry after 561.688859ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.405721 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0416 00:42:50.464159 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0416 00:42:50.496685 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.496715 369017 retry.go:31] will retry after 830.217116ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0416 00:42:50.540203 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.540301 369017 retry.go:31] will retry after 753.850811ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.541337 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0416 00:42:50.620859 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.620927 369017 retry.go:31] will retry after 1.219239665s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.854313 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0416 00:42:50.935888 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.935920 369017 retry.go:31] will retry after 491.996337ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:50.995614 369017 node_ready.go:53] error getting node "old-k8s-version-014065": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-014065": dial tcp 192.168.76.2:8443: connect: connection refused
I0416 00:42:51.295050 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:42:51.327532 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0416 00:42:51.394513 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.394598 369017 retry.go:31] will retry after 1.268284118s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0416 00:42:51.427769 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.427806 369017 retry.go:31] will retry after 849.929226ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.428856 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0416 00:42:51.503508 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.503546 369017 retry.go:31] will retry after 1.404901225s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.841160 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0416 00:42:51.991915 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:51.991995 369017 retry.go:31] will retry after 1.160838179s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:52.277919 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0416 00:42:52.381657 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:52.381696 369017 retry.go:31] will retry after 1.656338858s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:52.663522 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0416 00:42:52.846688 369017 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:52.846721 369017 retry.go:31] will retry after 2.009302959s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0416 00:42:52.908988 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:42:52.995952 369017 node_ready.go:53] error getting node "old-k8s-version-014065": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-014065": dial tcp 192.168.76.2:8443: connect: connection refused
I0416 00:42:53.153349 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0416 00:42:54.039038 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0416 00:42:54.856372 369017 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:43:00.919485 369017 node_ready.go:49] node "old-k8s-version-014065" has status "Ready":"True"
I0416 00:43:00.919508 369017 node_ready.go:38] duration metric: took 11.924597466s for node "old-k8s-version-014065" to be "Ready" ...
I0416 00:43:00.919518 369017 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0416 00:43:01.167938 369017 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-ftt5t" in "kube-system" namespace to be "Ready" ...
I0416 00:43:01.412694 369017 pod_ready.go:92] pod "coredns-74ff55c5b-ftt5t" in "kube-system" namespace has status "Ready":"True"
I0416 00:43:01.412771 369017 pod_ready.go:81] duration metric: took 232.943583ms for pod "coredns-74ff55c5b-ftt5t" in "kube-system" namespace to be "Ready" ...
I0416 00:43:01.412797 369017 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:43:01.513631 369017 pod_ready.go:92] pod "etcd-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"True"
I0416 00:43:01.513710 369017 pod_ready.go:81] duration metric: took 100.892073ms for pod "etcd-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:43:01.513738 369017 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:43:02.655486 369017 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (9.746450101s)
I0416 00:43:02.655584 369017 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (9.502201615s)
I0416 00:43:02.655824 369017 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-014065"
I0416 00:43:03.292436 369017 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (9.253350544s)
I0416 00:43:03.295982 369017 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-014065 addons enable metrics-server
I0416 00:43:03.292746 369017 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.436349034s)
I0416 00:43:03.300599 369017 out.go:177] * Enabled addons: metrics-server, default-storageclass, storage-provisioner, dashboard
I0416 00:43:03.302376 369017 addons.go:505] duration metric: took 14.531905703s for enable addons: enabled=[metrics-server default-storageclass storage-provisioner dashboard]
I0416 00:43:03.521246 369017 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:06.024246 369017 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"True"
I0416 00:43:06.024279 369017 pod_ready.go:81] duration metric: took 4.510497756s for pod "kube-apiserver-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:43:06.024293 369017 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:43:08.030195 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:10.033316 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:12.530165 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:14.531052 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:17.031906 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:19.611876 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:22.031623 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:24.032525 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:26.530649 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:28.530928 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:30.531241 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:32.531599 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:35.031841 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:37.032236 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:39.530653 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:41.531003 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:44.031100 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:46.532344 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:49.031085 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:51.031656 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:53.031817 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:55.035014 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:43:57.531030 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:00.142213 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:02.532405 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:05.032266 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:07.530814 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:10.038046 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:12.530831 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:14.530998 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:16.531449 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:18.532278 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:21.030934 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:23.032547 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:25.531445 369017 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:27.031107 369017 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"True"
I0416 00:44:27.031136 369017 pod_ready.go:81] duration metric: took 1m21.006834629s for pod "kube-controller-manager-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:44:27.031148 369017 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-2ltgk" in "kube-system" namespace to be "Ready" ...
I0416 00:44:27.037549 369017 pod_ready.go:92] pod "kube-proxy-2ltgk" in "kube-system" namespace has status "Ready":"True"
I0416 00:44:27.037633 369017 pod_ready.go:81] duration metric: took 6.456151ms for pod "kube-proxy-2ltgk" in "kube-system" namespace to be "Ready" ...
I0416 00:44:27.037649 369017 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:44:27.043793 369017 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-014065" in "kube-system" namespace has status "Ready":"True"
I0416 00:44:27.043816 369017 pod_ready.go:81] duration metric: took 6.159329ms for pod "kube-scheduler-old-k8s-version-014065" in "kube-system" namespace to be "Ready" ...
I0416 00:44:27.043829 369017 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace to be "Ready" ...
I0416 00:44:29.050672 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:31.549733 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:33.550168 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:36.051016 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:38.549215 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:40.551552 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:43.052505 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:45.063320 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:47.551081 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:50.050986 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:52.549641 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:54.550043 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:57.050475 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:44:59.051409 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:01.051769 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:03.550075 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:06.051560 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:08.549431 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:10.550492 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:13.050825 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:15.060133 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:17.550402 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:20.053548 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:22.550057 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:24.550163 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:27.051853 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:29.550263 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:32.050455 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:34.051181 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:36.054970 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:38.055630 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:40.550076 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:42.550172 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:45.056828 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:47.549401 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:50.052078 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:52.088288 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:54.550454 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:57.050133 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:45:59.052754 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:01.550182 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:04.050585 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:06.056869 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:08.549732 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:10.549942 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:12.550763 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:14.552330 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:17.051662 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:19.550206 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:22.049924 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:24.051476 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:26.550936 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:29.050835 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:31.549907 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:33.550519 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:36.050609 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:38.057436 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:40.550723 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:42.596930 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:45.067901 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:47.550192 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:49.550754 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:52.052757 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:54.550808 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:57.051042 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:46:59.052507 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:01.552591 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:04.050376 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:06.551219 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:09.056905 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:11.057732 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:13.550199 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:16.050496 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:18.051509 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:20.549970 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:22.550237 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:24.551772 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:27.051148 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:29.550234 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:32.051368 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:34.549963 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:36.551305 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:39.050909 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:41.060958 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:43.550338 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:45.551018 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:47.553300 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:50.050920 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:52.052167 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:54.550341 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:56.550842 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:58.553866 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:00.566529 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:03.055439 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:05.549612 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:08.050814 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:10.051128 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:12.550061 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:15.053238 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:17.553163 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:20.050973 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:22.053952 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:24.549677 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:26.550323 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:27.050917 369017 pod_ready.go:81] duration metric: took 4m0.007077036s for pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace to be "Ready" ...
E0416 00:48:27.050943 369017 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0416 00:48:27.050953 369017 pod_ready.go:38] duration metric: took 5m26.131424924s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0416 00:48:27.050972 369017 api_server.go:52] waiting for apiserver process to appear ...
I0416 00:48:27.051054 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0416 00:48:27.073537 369017 logs.go:276] 2 containers: [a7d7845d2402 b8ea3fa2ab02]
I0416 00:48:27.073627 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0416 00:48:27.092597 369017 logs.go:276] 2 containers: [33107d331e0b fd5230a8d74b]
I0416 00:48:27.092711 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0416 00:48:27.123060 369017 logs.go:276] 2 containers: [65e7340af5ef 697870ff99a4]
I0416 00:48:27.123158 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0416 00:48:27.141834 369017 logs.go:276] 2 containers: [2d7d1b9e8353 7b437d823755]
I0416 00:48:27.141920 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0416 00:48:27.160491 369017 logs.go:276] 2 containers: [fa54eb276fa9 bf3ceb2acadb]
I0416 00:48:27.160587 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0416 00:48:27.180907 369017 logs.go:276] 2 containers: [b3c3c455ea1c 4cc3ed1cf27e]
I0416 00:48:27.180992 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0416 00:48:27.198507 369017 logs.go:276] 0 containers: []
W0416 00:48:27.198530 369017 logs.go:278] No container was found matching "kindnet"
I0416 00:48:27.198587 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0416 00:48:27.216044 369017 logs.go:276] 1 containers: [c311fb93e11b]
I0416 00:48:27.216241 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0416 00:48:27.232689 369017 logs.go:276] 2 containers: [c5eb3f5fa95a d8e8f85e95c4]
I0416 00:48:27.232722 369017 logs.go:123] Gathering logs for kubernetes-dashboard [c311fb93e11b] ...
I0416 00:48:27.232735 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c311fb93e11b"
I0416 00:48:27.254742 369017 logs.go:123] Gathering logs for kube-apiserver [a7d7845d2402] ...
I0416 00:48:27.254816 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 a7d7845d2402"
I0416 00:48:27.299302 369017 logs.go:123] Gathering logs for coredns [65e7340af5ef] ...
I0416 00:48:27.299340 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 65e7340af5ef"
I0416 00:48:27.327427 369017 logs.go:123] Gathering logs for kube-scheduler [7b437d823755] ...
I0416 00:48:27.327461 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b437d823755"
I0416 00:48:27.354603 369017 logs.go:123] Gathering logs for kube-proxy [bf3ceb2acadb] ...
I0416 00:48:27.354636 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 bf3ceb2acadb"
I0416 00:48:27.377974 369017 logs.go:123] Gathering logs for storage-provisioner [c5eb3f5fa95a] ...
I0416 00:48:27.378013 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c5eb3f5fa95a"
I0416 00:48:27.400538 369017 logs.go:123] Gathering logs for storage-provisioner [d8e8f85e95c4] ...
I0416 00:48:27.400566 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 d8e8f85e95c4"
I0416 00:48:27.421475 369017 logs.go:123] Gathering logs for container status ...
I0416 00:48:27.421505 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0416 00:48:27.501109 369017 logs.go:123] Gathering logs for kubelet ...
I0416 00:48:27.501142 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0416 00:48:27.578790 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:03 old-k8s-version-014065 kubelet[1223]: E0416 00:43:03.441058 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.580653 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:04 old-k8s-version-014065 kubelet[1223]: E0416 00:43:04.639437 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.583270 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:15 old-k8s-version-014065 kubelet[1223]: E0416 00:43:15.964300 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.592833 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:20 old-k8s-version-014065 kubelet[1223]: E0416 00:43:20.969599 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.593059 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:21 old-k8s-version-014065 kubelet[1223]: E0416 00:43:21.832458 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.594338 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:26 old-k8s-version-014065 kubelet[1223]: E0416 00:43:26.931292 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.597710 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.428287 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.598545 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.976063 1223 pod_workers.go:191] Error syncing pod cef2692c-ceee-4c9c-892a-75dcaae5ab8a ("storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"
W0416 00:48:27.600747 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:38 old-k8s-version-014065 kubelet[1223]: E0416 00:43:38.955013 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.601295 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:46 old-k8s-version-014065 kubelet[1223]: E0416 00:43:46.931311 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.601614 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:53 old-k8s-version-014065 kubelet[1223]: E0416 00:43:53.963232 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.604001 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:02 old-k8s-version-014065 kubelet[1223]: E0416 00:44:02.430723 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.604206 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:07 old-k8s-version-014065 kubelet[1223]: E0416 00:44:07.936331 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.604402 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:14 old-k8s-version-014065 kubelet[1223]: E0416 00:44:14.931692 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.606653 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:20 old-k8s-version-014065 kubelet[1223]: E0416 00:44:20.968241 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.606863 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:25 old-k8s-version-014065 kubelet[1223]: E0416 00:44:25.952535 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607048 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:32 old-k8s-version-014065 kubelet[1223]: E0416 00:44:32.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607278 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:40 old-k8s-version-014065 kubelet[1223]: E0416 00:44:40.931086 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607466 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:46 old-k8s-version-014065 kubelet[1223]: E0416 00:44:46.931443 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.609801 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:54 old-k8s-version-014065 kubelet[1223]: E0416 00:44:54.366332 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.610002 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:01 old-k8s-version-014065 kubelet[1223]: E0416 00:45:01.931372 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610273 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:08 old-k8s-version-014065 kubelet[1223]: E0416 00:45:08.933619 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610493 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:12 old-k8s-version-014065 kubelet[1223]: E0416 00:45:12.931273 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610681 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.931035 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610917 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.953472 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.611117 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:35 old-k8s-version-014065 kubelet[1223]: E0416 00:45:35.932086 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.611363 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:38 old-k8s-version-014065 kubelet[1223]: E0416 00:45:38.937563 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.613506 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:49 old-k8s-version-014065 kubelet[1223]: E0416 00:45:49.959737 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.613707 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:50 old-k8s-version-014065 kubelet[1223]: E0416 00:45:50.931084 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.613892 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:02 old-k8s-version-014065 kubelet[1223]: E0416 00:46:02.934939 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.614087 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:04 old-k8s-version-014065 kubelet[1223]: E0416 00:46:04.931137 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.614275 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:14 old-k8s-version-014065 kubelet[1223]: E0416 00:46:14.942668 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.616488 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:16 old-k8s-version-014065 kubelet[1223]: E0416 00:46:16.403347 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.616677 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:27 old-k8s-version-014065 kubelet[1223]: E0416 00:46:27.930989 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.616873 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:30 old-k8s-version-014065 kubelet[1223]: E0416 00:46:30.931502 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617084 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:40 old-k8s-version-014065 kubelet[1223]: E0416 00:46:40.931581 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617280 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:45 old-k8s-version-014065 kubelet[1223]: E0416 00:46:45.971319 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617464 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:53 old-k8s-version-014065 kubelet[1223]: E0416 00:46:53.934295 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617670 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:56 old-k8s-version-014065 kubelet[1223]: E0416 00:46:56.930865 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617855 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:05 old-k8s-version-014065 kubelet[1223]: E0416 00:47:05.931075 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618050 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:10 old-k8s-version-014065 kubelet[1223]: E0416 00:47:10.931300 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618235 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:20 old-k8s-version-014065 kubelet[1223]: E0416 00:47:20.932206 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618431 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:25 old-k8s-version-014065 kubelet[1223]: E0416 00:47:25.931180 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618616 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:31 old-k8s-version-014065 kubelet[1223]: E0416 00:47:31.931916 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618840 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:37 old-k8s-version-014065 kubelet[1223]: E0416 00:47:37.931161 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619026 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:44 old-k8s-version-014065 kubelet[1223]: E0416 00:47:44.931376 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619351 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:48 old-k8s-version-014065 kubelet[1223]: E0416 00:47:48.937314 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619548 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619745 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619929 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.620123 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.620308 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0416 00:48:27.620319 369017 logs.go:123] Gathering logs for kube-apiserver [b8ea3fa2ab02] ...
I0416 00:48:27.620334 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b8ea3fa2ab02"
I0416 00:48:27.698206 369017 logs.go:123] Gathering logs for coredns [697870ff99a4] ...
I0416 00:48:27.698250 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 697870ff99a4"
I0416 00:48:27.720130 369017 logs.go:123] Gathering logs for kube-scheduler [2d7d1b9e8353] ...
I0416 00:48:27.720164 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 2d7d1b9e8353"
I0416 00:48:27.741457 369017 logs.go:123] Gathering logs for dmesg ...
I0416 00:48:27.741485 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0416 00:48:27.764244 369017 logs.go:123] Gathering logs for describe nodes ...
I0416 00:48:27.764279 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0416 00:48:27.930626 369017 logs.go:123] Gathering logs for kube-proxy [fa54eb276fa9] ...
I0416 00:48:27.930654 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fa54eb276fa9"
I0416 00:48:27.962973 369017 logs.go:123] Gathering logs for Docker ...
I0416 00:48:27.963046 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0416 00:48:27.999906 369017 logs.go:123] Gathering logs for etcd [33107d331e0b] ...
I0416 00:48:27.999940 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 33107d331e0b"
I0416 00:48:28.028776 369017 logs.go:123] Gathering logs for etcd [fd5230a8d74b] ...
I0416 00:48:28.028817 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fd5230a8d74b"
I0416 00:48:28.064939 369017 logs.go:123] Gathering logs for kube-controller-manager [b3c3c455ea1c] ...
I0416 00:48:28.064970 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b3c3c455ea1c"
I0416 00:48:28.117348 369017 logs.go:123] Gathering logs for kube-controller-manager [4cc3ed1cf27e] ...
I0416 00:48:28.117379 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4cc3ed1cf27e"
I0416 00:48:28.175750 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:28.175783 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0416 00:48:28.175846 369017 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0416 00:48:28.175861 369017 out.go:239] Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175875 369017 out.go:239] Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175885 369017 out.go:239] Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175896 369017 out.go:239] Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175904 369017 out.go:239] Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0416 00:48:28.175912 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:28.175920 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:48:38.176694 369017 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0416 00:48:38.191932 369017 api_server.go:72] duration metric: took 5m49.421692349s to wait for apiserver process to appear ...
I0416 00:48:38.191963 369017 api_server.go:88] waiting for apiserver healthz status ...
I0416 00:48:38.192051 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0416 00:48:38.209623 369017 logs.go:276] 2 containers: [a7d7845d2402 b8ea3fa2ab02]
I0416 00:48:38.209701 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0416 00:48:38.228837 369017 logs.go:276] 2 containers: [33107d331e0b fd5230a8d74b]
I0416 00:48:38.228918 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0416 00:48:38.245852 369017 logs.go:276] 2 containers: [65e7340af5ef 697870ff99a4]
I0416 00:48:38.245943 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0416 00:48:38.268054 369017 logs.go:276] 2 containers: [2d7d1b9e8353 7b437d823755]
I0416 00:48:38.268136 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0416 00:48:38.292495 369017 logs.go:276] 2 containers: [fa54eb276fa9 bf3ceb2acadb]
I0416 00:48:38.292571 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0416 00:48:38.314076 369017 logs.go:276] 2 containers: [b3c3c455ea1c 4cc3ed1cf27e]
I0416 00:48:38.314160 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0416 00:48:38.330761 369017 logs.go:276] 0 containers: []
W0416 00:48:38.330781 369017 logs.go:278] No container was found matching "kindnet"
I0416 00:48:38.330834 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0416 00:48:38.349329 369017 logs.go:276] 2 containers: [c5eb3f5fa95a d8e8f85e95c4]
I0416 00:48:38.349479 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0416 00:48:38.366973 369017 logs.go:276] 1 containers: [c311fb93e11b]
I0416 00:48:38.367003 369017 logs.go:123] Gathering logs for kube-apiserver [b8ea3fa2ab02] ...
I0416 00:48:38.367015 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b8ea3fa2ab02"
I0416 00:48:38.422460 369017 logs.go:123] Gathering logs for etcd [33107d331e0b] ...
I0416 00:48:38.422494 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 33107d331e0b"
I0416 00:48:38.459188 369017 logs.go:123] Gathering logs for kube-scheduler [2d7d1b9e8353] ...
I0416 00:48:38.459287 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 2d7d1b9e8353"
I0416 00:48:38.488818 369017 logs.go:123] Gathering logs for kube-controller-manager [b3c3c455ea1c] ...
I0416 00:48:38.488856 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b3c3c455ea1c"
I0416 00:48:38.528135 369017 logs.go:123] Gathering logs for kubelet ...
I0416 00:48:38.528170 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0416 00:48:38.584942 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:03 old-k8s-version-014065 kubelet[1223]: E0416 00:43:03.441058 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.586673 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:04 old-k8s-version-014065 kubelet[1223]: E0416 00:43:04.639437 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.589065 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:15 old-k8s-version-014065 kubelet[1223]: E0416 00:43:15.964300 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.593156 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:20 old-k8s-version-014065 kubelet[1223]: E0416 00:43:20.969599 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.593356 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:21 old-k8s-version-014065 kubelet[1223]: E0416 00:43:21.832458 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.594043 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:26 old-k8s-version-014065 kubelet[1223]: E0416 00:43:26.931292 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.596270 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.428287 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.597041 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.976063 1223 pod_workers.go:191] Error syncing pod cef2692c-ceee-4c9c-892a-75dcaae5ab8a ("storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"
W0416 00:48:38.599100 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:38 old-k8s-version-014065 kubelet[1223]: E0416 00:43:38.955013 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.599665 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:46 old-k8s-version-014065 kubelet[1223]: E0416 00:43:46.931311 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.599979 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:53 old-k8s-version-014065 kubelet[1223]: E0416 00:43:53.963232 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.602177 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:02 old-k8s-version-014065 kubelet[1223]: E0416 00:44:02.430723 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.602364 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:07 old-k8s-version-014065 kubelet[1223]: E0416 00:44:07.936331 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.602559 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:14 old-k8s-version-014065 kubelet[1223]: E0416 00:44:14.931692 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.604596 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:20 old-k8s-version-014065 kubelet[1223]: E0416 00:44:20.968241 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.604793 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:25 old-k8s-version-014065 kubelet[1223]: E0416 00:44:25.952535 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.604977 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:32 old-k8s-version-014065 kubelet[1223]: E0416 00:44:32.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.605172 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:40 old-k8s-version-014065 kubelet[1223]: E0416 00:44:40.931086 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.605356 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:46 old-k8s-version-014065 kubelet[1223]: E0416 00:44:46.931443 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.607588 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:54 old-k8s-version-014065 kubelet[1223]: E0416 00:44:54.366332 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.607775 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:01 old-k8s-version-014065 kubelet[1223]: E0416 00:45:01.931372 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.607984 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:08 old-k8s-version-014065 kubelet[1223]: E0416 00:45:08.933619 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608167 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:12 old-k8s-version-014065 kubelet[1223]: E0416 00:45:12.931273 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608349 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.931035 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608545 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.953472 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608728 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:35 old-k8s-version-014065 kubelet[1223]: E0416 00:45:35.932086 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608930 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:38 old-k8s-version-014065 kubelet[1223]: E0416 00:45:38.937563 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.610973 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:49 old-k8s-version-014065 kubelet[1223]: E0416 00:45:49.959737 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.611166 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:50 old-k8s-version-014065 kubelet[1223]: E0416 00:45:50.931084 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611382 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:02 old-k8s-version-014065 kubelet[1223]: E0416 00:46:02.934939 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611578 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:04 old-k8s-version-014065 kubelet[1223]: E0416 00:46:04.931137 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611762 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:14 old-k8s-version-014065 kubelet[1223]: E0416 00:46:14.942668 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.613954 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:16 old-k8s-version-014065 kubelet[1223]: E0416 00:46:16.403347 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.614138 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:27 old-k8s-version-014065 kubelet[1223]: E0416 00:46:27.930989 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614333 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:30 old-k8s-version-014065 kubelet[1223]: E0416 00:46:30.931502 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614519 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:40 old-k8s-version-014065 kubelet[1223]: E0416 00:46:40.931581 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614712 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:45 old-k8s-version-014065 kubelet[1223]: E0416 00:46:45.971319 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614911 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:53 old-k8s-version-014065 kubelet[1223]: E0416 00:46:53.934295 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615106 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:56 old-k8s-version-014065 kubelet[1223]: E0416 00:46:56.930865 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615296 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:05 old-k8s-version-014065 kubelet[1223]: E0416 00:47:05.931075 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615490 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:10 old-k8s-version-014065 kubelet[1223]: E0416 00:47:10.931300 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615673 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:20 old-k8s-version-014065 kubelet[1223]: E0416 00:47:20.932206 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615866 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:25 old-k8s-version-014065 kubelet[1223]: E0416 00:47:25.931180 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616049 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:31 old-k8s-version-014065 kubelet[1223]: E0416 00:47:31.931916 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616243 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:37 old-k8s-version-014065 kubelet[1223]: E0416 00:47:37.931161 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616448 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:44 old-k8s-version-014065 kubelet[1223]: E0416 00:47:44.931376 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616644 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:48 old-k8s-version-014065 kubelet[1223]: E0416 00:47:48.937314 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616827 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617022 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617204 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617401 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617586 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617782 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0416 00:48:38.617792 369017 logs.go:123] Gathering logs for kube-apiserver [a7d7845d2402] ...
I0416 00:48:38.617805 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 a7d7845d2402"
I0416 00:48:38.659232 369017 logs.go:123] Gathering logs for coredns [65e7340af5ef] ...
I0416 00:48:38.659348 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 65e7340af5ef"
I0416 00:48:38.688264 369017 logs.go:123] Gathering logs for kube-proxy [fa54eb276fa9] ...
I0416 00:48:38.688293 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fa54eb276fa9"
I0416 00:48:38.708927 369017 logs.go:123] Gathering logs for kube-proxy [bf3ceb2acadb] ...
I0416 00:48:38.708955 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 bf3ceb2acadb"
I0416 00:48:38.729344 369017 logs.go:123] Gathering logs for kube-controller-manager [4cc3ed1cf27e] ...
I0416 00:48:38.729374 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4cc3ed1cf27e"
I0416 00:48:38.771163 369017 logs.go:123] Gathering logs for Docker ...
I0416 00:48:38.771228 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0416 00:48:38.802002 369017 logs.go:123] Gathering logs for dmesg ...
I0416 00:48:38.802041 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0416 00:48:38.822471 369017 logs.go:123] Gathering logs for describe nodes ...
I0416 00:48:38.822500 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0416 00:48:38.978993 369017 logs.go:123] Gathering logs for container status ...
I0416 00:48:38.979023 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0416 00:48:39.074662 369017 logs.go:123] Gathering logs for coredns [697870ff99a4] ...
I0416 00:48:39.074703 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 697870ff99a4"
I0416 00:48:39.100511 369017 logs.go:123] Gathering logs for storage-provisioner [d8e8f85e95c4] ...
I0416 00:48:39.100545 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 d8e8f85e95c4"
I0416 00:48:39.122784 369017 logs.go:123] Gathering logs for storage-provisioner [c5eb3f5fa95a] ...
I0416 00:48:39.122813 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c5eb3f5fa95a"
I0416 00:48:39.144632 369017 logs.go:123] Gathering logs for kubernetes-dashboard [c311fb93e11b] ...
I0416 00:48:39.144662 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c311fb93e11b"
I0416 00:48:39.166870 369017 logs.go:123] Gathering logs for etcd [fd5230a8d74b] ...
I0416 00:48:39.166901 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fd5230a8d74b"
I0416 00:48:39.190433 369017 logs.go:123] Gathering logs for kube-scheduler [7b437d823755] ...
I0416 00:48:39.190464 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b437d823755"
I0416 00:48:39.213048 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:39.213073 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0416 00:48:39.213162 369017 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0416 00:48:39.213174 369017 out.go:239] Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213185 369017 out.go:239] Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213207 369017 out.go:239] Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213217 369017 out.go:239] Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213223 369017 out.go:239] Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0416 00:48:39.213235 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:39.213242 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:48:49.213969 369017 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0416 00:48:49.227100 369017 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0416 00:48:49.230281 369017 out.go:177]
W0416 00:48:49.232609 369017 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0416 00:48:49.232667 369017 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0416 00:48:49.232693 369017 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0416 00:48:49.232702 369017 out.go:239] *
*
W0416 00:48:49.234077 369017 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0416 00:48:49.236416 369017 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-014065 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-014065
helpers_test.go:235: (dbg) docker inspect old-k8s-version-014065:
-- stdout --
[
{
"Id": "452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60",
"Created": "2024-04-16T00:39:40.987999443Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 369218,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-04-16T00:42:41.221775429Z",
"FinishedAt": "2024-04-16T00:42:40.122892043Z"
},
"Image": "sha256:05b5b2cbc7157bfe11e03d0beeaf25e36e83e7ad2b499390548ca8693c4ec20b",
"ResolvConfPath": "/var/lib/docker/containers/452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60/hostname",
"HostsPath": "/var/lib/docker/containers/452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60/hosts",
"LogPath": "/var/lib/docker/containers/452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60/452df2b37af5461b3847af989f157738d2b001fb3a94037a0b53ad327cfc3f60-json.log",
"Name": "/old-k8s-version-014065",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-014065:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-014065",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/e5e795d5efe9e3395660beeb319a913dd91648822a94033fc56d569f00ea12dd-init/diff:/var/lib/docker/overlay2/d2fb7d5dfad483877edf794e760fbf311a1d68be07bb2438f714c78875e64b61/diff",
"MergedDir": "/var/lib/docker/overlay2/e5e795d5efe9e3395660beeb319a913dd91648822a94033fc56d569f00ea12dd/merged",
"UpperDir": "/var/lib/docker/overlay2/e5e795d5efe9e3395660beeb319a913dd91648822a94033fc56d569f00ea12dd/diff",
"WorkDir": "/var/lib/docker/overlay2/e5e795d5efe9e3395660beeb319a913dd91648822a94033fc56d569f00ea12dd/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-014065",
"Source": "/var/lib/docker/volumes/old-k8s-version-014065/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-014065",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-014065",
"name.minikube.sigs.k8s.io": "old-k8s-version-014065",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "2366f76f50746089ddf80f87abc9f131724ad0cbdf337794fed5772b35677fe5",
"SandboxKey": "/var/run/docker/netns/2366f76f5074",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33137"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33136"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33133"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33135"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33134"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-014065": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"NetworkID": "5628bbc7652a15748e66b46ffe1774aed037ef9be513caab37abce93f2de8a59",
"EndpointID": "1a0e629a9588566a74c3a22b9431a54d443b79b273903fb5d604cd92557c3516",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-014065",
"452df2b37af5"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-014065 -n old-k8s-version-014065
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-014065 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-014065 logs -n 25: (1.482671767s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|------------------------|---------|----------------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|------------------------|---------|----------------|---------------------|---------------------|
| ssh | -p kubenet-128493 sudo | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:40 UTC |
| | containerd config dump | | | | | |
| ssh | -p kubenet-128493 sudo | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | |
| | systemctl status crio --all | | | | | |
| | --full --no-pager | | | | | |
| ssh | -p kubenet-128493 sudo | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:40 UTC |
| | systemctl cat crio --no-pager | | | | | |
| ssh | -p kubenet-128493 sudo find | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:40 UTC |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p kubenet-128493 sudo crio | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:40 UTC |
| | config | | | | | |
| delete | -p kubenet-128493 | kubenet-128493 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:40 UTC |
| start | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:40 UTC | 16 Apr 24 00:41 UTC |
| | --memory=2200 --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.30.0-rc.2 | | | | | |
| addons | enable metrics-server -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:41 UTC | 16 Apr 24 00:41 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:41 UTC | 16 Apr 24 00:41 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:41 UTC | 16 Apr 24 00:41 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:41 UTC | 16 Apr 24 00:46 UTC |
| | --memory=2200 --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.30.0-rc.2 | | | | | |
| addons | enable metrics-server -p old-k8s-version-014065 | old-k8s-version-014065 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:42 UTC | 16 Apr 24 00:42 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-014065 | old-k8s-version-014065 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:42 UTC | 16 Apr 24 00:42 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-014065 | old-k8s-version-014065 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:42 UTC | 16 Apr 24 00:42 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-014065 | old-k8s-version-014065 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:42 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| image | no-preload-140300 image list | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:46 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:46 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:46 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:46 UTC |
| delete | -p no-preload-140300 | no-preload-140300 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:46 UTC |
| start | -p embed-certs-534050 | embed-certs-534050 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:46 UTC | 16 Apr 24 00:47 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.29.3 | | | | | |
| addons | enable metrics-server -p embed-certs-534050 | embed-certs-534050 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:47 UTC | 16 Apr 24 00:47 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p embed-certs-534050 | embed-certs-534050 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:47 UTC | 16 Apr 24 00:47 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p embed-certs-534050 | embed-certs-534050 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:47 UTC | 16 Apr 24 00:47 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p embed-certs-534050 | embed-certs-534050 | jenkins | v1.33.0-beta.0 | 16 Apr 24 00:47 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.29.3 | | | | | |
|---------|--------------------------------------------------------|------------------------|---------|----------------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/04/16 00:47:52
Running on machine: ip-172-31-30-239
Binary: Built with gc go1.22.1 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0416 00:47:52.169928 381606 out.go:291] Setting OutFile to fd 1 ...
I0416 00:47:52.170078 381606 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:47:52.170088 381606 out.go:304] Setting ErrFile to fd 2...
I0416 00:47:52.170094 381606 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:47:52.170419 381606 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18647-2210/.minikube/bin
I0416 00:47:52.170957 381606 out.go:298] Setting JSON to false
I0416 00:47:52.172162 381606 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":5408,"bootTime":1713223065,"procs":296,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1057-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0416 00:47:52.172258 381606 start.go:139] virtualization:
I0416 00:47:52.176023 381606 out.go:177] * [embed-certs-534050] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0416 00:47:52.178365 381606 out.go:177] - MINIKUBE_LOCATION=18647
I0416 00:47:52.180277 381606 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0416 00:47:52.178433 381606 notify.go:220] Checking for updates...
I0416 00:47:52.184518 381606 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:47:52.186737 381606 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18647-2210/.minikube
I0416 00:47:52.188750 381606 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0416 00:47:52.190857 381606 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0416 00:47:52.193394 381606 config.go:182] Loaded profile config "embed-certs-534050": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.29.3
I0416 00:47:52.193917 381606 driver.go:392] Setting default libvirt URI to qemu:///system
I0416 00:47:52.214700 381606 docker.go:122] docker version: linux-26.0.1:Docker Engine - Community
I0416 00:47:52.214893 381606 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0416 00:47:52.280238 381606 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:40 OomKillDisable:true NGoroutines:52 SystemTime:2024-04-16 00:47:52.262737594 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1057-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.26.1]] Warnings:<nil>}}
I0416 00:47:52.280350 381606 docker.go:295] overlay module found
I0416 00:47:52.283601 381606 out.go:177] * Using the docker driver based on existing profile
I0416 00:47:52.285713 381606 start.go:297] selected driver: docker
I0416 00:47:52.285734 381606 start.go:901] validating driver "docker" against &{Name:embed-certs-534050 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-534050 Namespace:default APIServerHAVIP: APIServerN
ame:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:
/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:47:52.285849 381606 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0416 00:47:52.286491 381606 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0416 00:47:52.340629 381606 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:40 OomKillDisable:true NGoroutines:52 SystemTime:2024-04-16 00:47:52.329683207 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1057-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:26.0.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.26.1]] Warnings:<nil>}}
I0416 00:47:52.341013 381606 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0416 00:47:52.341074 381606 cni.go:84] Creating CNI manager for ""
I0416 00:47:52.341095 381606 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I0416 00:47:52.341148 381606 start.go:340] cluster config:
{Name:embed-certs-534050 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-534050 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: Mo
untMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:47:52.344457 381606 out.go:177] * Starting "embed-certs-534050" primary control-plane node in "embed-certs-534050" cluster
I0416 00:47:52.346204 381606 cache.go:121] Beginning downloading kic base image for docker with docker
I0416 00:47:52.348384 381606 out.go:177] * Pulling base image v0.0.43-1713215244-18647 ...
I0416 00:47:52.350523 381606 preload.go:132] Checking if preload exists for k8s version v1.29.3 and runtime docker
I0416 00:47:52.350575 381606 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18647-2210/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-docker-overlay2-arm64.tar.lz4
I0416 00:47:52.350583 381606 cache.go:56] Caching tarball of preloaded images
I0416 00:47:52.350671 381606 preload.go:173] Found /home/jenkins/minikube-integration/18647-2210/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.3-docker-overlay2-arm64.tar.lz4 in cache, skipping download
I0416 00:47:52.350689 381606 cache.go:59] Finished verifying existence of preloaded tar for v1.29.3 on docker
I0416 00:47:52.350816 381606 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/config.json ...
I0416 00:47:52.351046 381606 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af in local docker daemon
I0416 00:47:52.367593 381606 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af in local docker daemon, skipping pull
I0416 00:47:52.367620 381606 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af exists in daemon, skipping load
I0416 00:47:52.367643 381606 cache.go:194] Successfully downloaded all kic artifacts
I0416 00:47:52.367679 381606 start.go:360] acquireMachinesLock for embed-certs-534050: {Name:mke7c4d31de3c5169ff1da2f0f005f363b817435 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0416 00:47:52.367749 381606 start.go:364] duration metric: took 43.569µs to acquireMachinesLock for "embed-certs-534050"
I0416 00:47:52.367774 381606 start.go:96] Skipping create...Using existing machine configuration
I0416 00:47:52.367780 381606 fix.go:54] fixHost starting:
I0416 00:47:52.368106 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:47:52.383637 381606 fix.go:112] recreateIfNeeded on embed-certs-534050: state=Stopped err=<nil>
W0416 00:47:52.383676 381606 fix.go:138] unexpected machine state, will restart: <nil>
I0416 00:47:52.386107 381606 out.go:177] * Restarting existing docker container for "embed-certs-534050" ...
I0416 00:47:52.052167 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:54.550341 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:52.388448 381606 cli_runner.go:164] Run: docker start embed-certs-534050
I0416 00:47:52.750077 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:47:52.769708 381606 kic.go:430] container "embed-certs-534050" state is running.
I0416 00:47:52.770094 381606 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-534050
I0416 00:47:52.792735 381606 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/config.json ...
I0416 00:47:52.792968 381606 machine.go:94] provisionDockerMachine start ...
I0416 00:47:52.793036 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:52.810656 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:52.810941 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:52.810957 381606 main.go:141] libmachine: About to run SSH command:
hostname
I0416 00:47:52.812468 381606 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:46570->127.0.0.1:33147: read: connection reset by peer
I0416 00:47:55.963509 381606 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-534050
I0416 00:47:55.963577 381606 ubuntu.go:169] provisioning hostname "embed-certs-534050"
I0416 00:47:55.963659 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:55.981352 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:55.981625 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:55.981643 381606 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-534050 && echo "embed-certs-534050" | sudo tee /etc/hostname
I0416 00:47:56.144711 381606 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-534050
I0416 00:47:56.144800 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:56.161425 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:56.161682 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:56.161704 381606 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-534050' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-534050/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-534050' | sudo tee -a /etc/hosts;
fi
fi
I0416 00:47:56.307437 381606 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:47:56.307465 381606 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18647-2210/.minikube CaCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18647-2210/.minikube}
I0416 00:47:56.307495 381606 ubuntu.go:177] setting up certificates
I0416 00:47:56.307504 381606 provision.go:84] configureAuth start
I0416 00:47:56.307565 381606 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-534050
I0416 00:47:56.323463 381606 provision.go:143] copyHostCerts
I0416 00:47:56.323533 381606 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem, removing ...
I0416 00:47:56.323543 381606 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem
I0416 00:47:56.323627 381606 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/key.pem (1679 bytes)
I0416 00:47:56.323733 381606 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem, removing ...
I0416 00:47:56.323738 381606 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem
I0416 00:47:56.323765 381606 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/ca.pem (1078 bytes)
I0416 00:47:56.323820 381606 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem, removing ...
I0416 00:47:56.323824 381606 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem
I0416 00:47:56.323848 381606 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18647-2210/.minikube/cert.pem (1123 bytes)
I0416 00:47:56.323893 381606 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem org=jenkins.embed-certs-534050 san=[127.0.0.1 192.168.85.2 embed-certs-534050 localhost minikube]
I0416 00:47:56.616169 381606 provision.go:177] copyRemoteCerts
I0416 00:47:56.616280 381606 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0416 00:47:56.616340 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:56.632887 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:47:56.740671 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0416 00:47:56.765705 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server.pem --> /etc/docker/server.pem (1224 bytes)
I0416 00:47:56.791632 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0416 00:47:56.822113 381606 provision.go:87] duration metric: took 514.596532ms to configureAuth
I0416 00:47:56.822138 381606 ubuntu.go:193] setting minikube options for container-runtime
I0416 00:47:56.822337 381606 config.go:182] Loaded profile config "embed-certs-534050": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.29.3
I0416 00:47:56.822394 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:56.839620 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:56.839884 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:56.839899 381606 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0416 00:47:56.984216 381606 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0416 00:47:56.984238 381606 ubuntu.go:71] root file system type: overlay
I0416 00:47:56.984376 381606 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0416 00:47:56.984451 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.000095 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:57.000355 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:57.000440 381606 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0416 00:47:57.172406 381606 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0416 00:47:57.173738 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.191362 381606 main.go:141] libmachine: Using SSH client type: native
I0416 00:47:57.191609 381606 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e90] 0x3e46f0 <nil> [] 0s} 127.0.0.1 33147 <nil> <nil>}
I0416 00:47:57.191635 381606 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0416 00:47:57.346006 381606 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:47:57.346034 381606 machine.go:97] duration metric: took 4.553049482s to provisionDockerMachine
I0416 00:47:57.346045 381606 start.go:293] postStartSetup for "embed-certs-534050" (driver="docker")
I0416 00:47:57.346057 381606 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0416 00:47:57.346129 381606 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0416 00:47:57.346176 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.362343 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:47:57.464168 381606 ssh_runner.go:195] Run: cat /etc/os-release
I0416 00:47:57.467394 381606 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0416 00:47:57.467434 381606 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0416 00:47:57.467445 381606 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0416 00:47:57.467451 381606 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0416 00:47:57.467461 381606 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-2210/.minikube/addons for local assets ...
I0416 00:47:57.467524 381606 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-2210/.minikube/files for local assets ...
I0416 00:47:57.467609 381606 filesync.go:149] local asset: /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem -> 75632.pem in /etc/ssl/certs
I0416 00:47:57.467714 381606 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0416 00:47:57.476443 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem --> /etc/ssl/certs/75632.pem (1708 bytes)
I0416 00:47:57.502190 381606 start.go:296] duration metric: took 156.129708ms for postStartSetup
I0416 00:47:57.502274 381606 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0416 00:47:57.502314 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.518741 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:47:57.616915 381606 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0416 00:47:57.621758 381606 fix.go:56] duration metric: took 5.253970982s for fixHost
I0416 00:47:57.621782 381606 start.go:83] releasing machines lock for "embed-certs-534050", held for 5.254019999s
I0416 00:47:57.621862 381606 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-534050
I0416 00:47:57.641710 381606 ssh_runner.go:195] Run: cat /version.json
I0416 00:47:57.641758 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.642025 381606 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0416 00:47:57.642082 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:47:57.660069 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:47:57.667521 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:47:57.880417 381606 ssh_runner.go:195] Run: systemctl --version
I0416 00:47:57.885125 381606 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0416 00:47:57.889499 381606 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0416 00:47:57.909899 381606 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0416 00:47:57.910026 381606 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0416 00:47:57.919939 381606 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0416 00:47:57.919965 381606 start.go:494] detecting cgroup driver to use...
I0416 00:47:57.920022 381606 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0416 00:47:57.920144 381606 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0416 00:47:57.941340 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I0416 00:47:57.951709 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0416 00:47:57.961474 381606 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0416 00:47:57.961553 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0416 00:47:57.971619 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0416 00:47:57.982854 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0416 00:47:57.993417 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0416 00:47:58.005858 381606 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0416 00:47:58.019395 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0416 00:47:58.031076 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0416 00:47:58.042587 381606 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0416 00:47:58.054831 381606 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0416 00:47:58.064755 381606 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0416 00:47:58.074075 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:47:58.184876 381606 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0416 00:47:58.334388 381606 start.go:494] detecting cgroup driver to use...
I0416 00:47:58.334442 381606 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0416 00:47:58.334514 381606 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0416 00:47:58.350505 381606 cruntime.go:279] skipping containerd shutdown because we are bound to it
I0416 00:47:58.350578 381606 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0416 00:47:58.365207 381606 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0416 00:47:58.387174 381606 ssh_runner.go:195] Run: which cri-dockerd
I0416 00:47:58.392559 381606 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0416 00:47:58.443177 381606 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0416 00:47:58.510393 381606 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0416 00:47:58.644705 381606 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0416 00:47:58.759994 381606 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0416 00:47:58.760191 381606 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0416 00:47:58.790109 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:47:58.904942 381606 ssh_runner.go:195] Run: sudo systemctl restart docker
I0416 00:47:59.384619 381606 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0416 00:47:59.397535 381606 ssh_runner.go:195] Run: sudo systemctl stop cri-docker.socket
I0416 00:47:59.410994 381606 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0416 00:47:59.424338 381606 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0416 00:47:59.519576 381606 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0416 00:47:59.609678 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:47:59.716922 381606 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0416 00:47:59.732697 381606 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0416 00:47:59.745390 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:47:59.838038 381606 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0416 00:47:59.921558 381606 start.go:541] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0416 00:47:59.921637 381606 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0416 00:47:59.925870 381606 start.go:562] Will wait 60s for crictl version
I0416 00:47:59.925934 381606 ssh_runner.go:195] Run: which crictl
I0416 00:47:59.930521 381606 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0416 00:47:59.974451 381606 start.go:578] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 26.0.1
RuntimeApiVersion: v1
I0416 00:47:59.974535 381606 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0416 00:47:59.995000 381606 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0416 00:47:56.550842 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:47:58.553866 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:00.566529 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:00.069740 381606 out.go:204] * Preparing Kubernetes v1.29.3 on Docker 26.0.1 ...
I0416 00:48:00.069856 381606 cli_runner.go:164] Run: docker network inspect embed-certs-534050 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0416 00:48:00.152217 381606 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0416 00:48:00.182338 381606 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:48:00.218610 381606 kubeadm.go:877] updating cluster {Name:embed-certs-534050 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-534050 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/mi
nikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0416 00:48:00.218771 381606 preload.go:132] Checking if preload exists for k8s version v1.29.3 and runtime docker
I0416 00:48:00.218848 381606 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0416 00:48:00.265211 381606 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.29.3
registry.k8s.io/kube-scheduler:v1.29.3
registry.k8s.io/kube-controller-manager:v1.29.3
registry.k8s.io/kube-proxy:v1.29.3
registry.k8s.io/etcd:3.5.12-0
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/pause:3.9
gcr.io/k8s-minikube/storage-provisioner:v5
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0416 00:48:00.265235 381606 docker.go:615] Images already preloaded, skipping extraction
I0416 00:48:00.265313 381606 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0416 00:48:00.339184 381606 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.29.3
registry.k8s.io/kube-controller-manager:v1.29.3
registry.k8s.io/kube-proxy:v1.29.3
registry.k8s.io/kube-scheduler:v1.29.3
registry.k8s.io/etcd:3.5.12-0
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/pause:3.9
gcr.io/k8s-minikube/storage-provisioner:v5
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0416 00:48:00.339252 381606 cache_images.go:84] Images are preloaded, skipping loading
I0416 00:48:00.339266 381606 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.29.3 docker true true} ...
I0416 00:48:00.339422 381606 kubeadm.go:940] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.29.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-534050 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.29.3 ClusterName:embed-certs-534050 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0416 00:48:00.339552 381606 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0416 00:48:00.538835 381606 cni.go:84] Creating CNI manager for ""
I0416 00:48:00.538867 381606 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I0416 00:48:00.538879 381606 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0416 00:48:00.538900 381606 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.29.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-534050 NodeName:embed-certs-534050 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:
/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/cri-dockerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0416 00:48:00.539328 381606 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/cri-dockerd.sock
name: "embed-certs-534050"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.29.3
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/cri-dockerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0416 00:48:00.539418 381606 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.29.3
I0416 00:48:00.570533 381606 binaries.go:44] Found k8s binaries, skipping transfer
I0416 00:48:00.570626 381606 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0416 00:48:00.587624 381606 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (317 bytes)
I0416 00:48:00.623623 381606 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0416 00:48:00.668285 381606 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2160 bytes)
I0416 00:48:00.691499 381606 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0416 00:48:00.697919 381606 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:48:00.711437 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:48:00.810658 381606 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0416 00:48:00.826240 381606 certs.go:68] Setting up /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050 for IP: 192.168.85.2
I0416 00:48:00.826266 381606 certs.go:194] generating shared ca certs ...
I0416 00:48:00.826286 381606 certs.go:226] acquiring lock for ca certs: {Name:mk0f2c276f9ccc821c50906b5561fa26a27a6ffe Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:48:00.826427 381606 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18647-2210/.minikube/ca.key
I0416 00:48:00.826474 381606 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.key
I0416 00:48:00.826485 381606 certs.go:256] generating profile certs ...
I0416 00:48:00.826578 381606 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/client.key
I0416 00:48:00.826670 381606 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/apiserver.key.4028827e
I0416 00:48:00.826748 381606 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/proxy-client.key
I0416 00:48:00.826864 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563.pem (1338 bytes)
W0416 00:48:00.826896 381606 certs.go:480] ignoring /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563_empty.pem, impossibly tiny 0 bytes
I0416 00:48:00.826917 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca-key.pem (1679 bytes)
I0416 00:48:00.826949 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/ca.pem (1078 bytes)
I0416 00:48:00.826979 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/cert.pem (1123 bytes)
I0416 00:48:00.827007 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/certs/key.pem (1679 bytes)
I0416 00:48:00.827055 381606 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem (1708 bytes)
I0416 00:48:00.827783 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0416 00:48:00.864291 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0416 00:48:00.894741 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0416 00:48:00.926271 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0416 00:48:00.963387 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0416 00:48:00.998201 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0416 00:48:01.035005 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0416 00:48:01.073085 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/profiles/embed-certs-534050/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0416 00:48:01.101296 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/files/etc/ssl/certs/75632.pem --> /usr/share/ca-certificates/75632.pem (1708 bytes)
I0416 00:48:01.157486 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0416 00:48:01.187316 381606 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-2210/.minikube/certs/7563.pem --> /usr/share/ca-certificates/7563.pem (1338 bytes)
I0416 00:48:01.224986 381606 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0416 00:48:01.247624 381606 ssh_runner.go:195] Run: openssl version
I0416 00:48:01.255879 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/75632.pem && ln -fs /usr/share/ca-certificates/75632.pem /etc/ssl/certs/75632.pem"
I0416 00:48:01.270235 381606 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/75632.pem
I0416 00:48:01.274707 381606 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Apr 15 23:43 /usr/share/ca-certificates/75632.pem
I0416 00:48:01.274795 381606 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/75632.pem
I0416 00:48:01.283107 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/75632.pem /etc/ssl/certs/3ec20f2e.0"
I0416 00:48:01.294290 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0416 00:48:01.305451 381606 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0416 00:48:01.309509 381606 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Apr 15 23:38 /usr/share/ca-certificates/minikubeCA.pem
I0416 00:48:01.309663 381606 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0416 00:48:01.317179 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0416 00:48:01.328021 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/7563.pem && ln -fs /usr/share/ca-certificates/7563.pem /etc/ssl/certs/7563.pem"
I0416 00:48:01.337904 381606 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/7563.pem
I0416 00:48:01.343167 381606 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Apr 15 23:43 /usr/share/ca-certificates/7563.pem
I0416 00:48:01.343266 381606 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/7563.pem
I0416 00:48:01.350761 381606 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/7563.pem /etc/ssl/certs/51391683.0"
I0416 00:48:01.361337 381606 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0416 00:48:01.365278 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0416 00:48:01.372452 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0416 00:48:01.379792 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0416 00:48:01.386734 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0416 00:48:01.393998 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0416 00:48:01.401421 381606 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0416 00:48:01.408660 381606 kubeadm.go:391] StartCluster: {Name:embed-certs-534050 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.3 ClusterName:embed-certs-534050 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minik
ube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:48:01.408847 381606 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0416 00:48:01.425623 381606 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0416 00:48:01.435555 381606 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0416 00:48:01.435575 381606 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0416 00:48:01.435581 381606 kubeadm.go:587] restartPrimaryControlPlane start ...
I0416 00:48:01.435648 381606 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0416 00:48:01.444915 381606 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0416 00:48:01.445548 381606 kubeconfig.go:47] verify endpoint returned: get endpoint: "embed-certs-534050" does not appear in /home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:48:01.445847 381606 kubeconfig.go:62] /home/jenkins/minikube-integration/18647-2210/kubeconfig needs updating (will repair): [kubeconfig missing "embed-certs-534050" cluster setting kubeconfig missing "embed-certs-534050" context setting]
I0416 00:48:01.446320 381606 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-2210/kubeconfig: {Name:mk2a4b2f2d98970b43b7e481fd26cc76bda92838 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:48:01.447723 381606 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0416 00:48:01.458144 381606 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.85.2
I0416 00:48:01.458175 381606 kubeadm.go:591] duration metric: took 22.588873ms to restartPrimaryControlPlane
I0416 00:48:01.458184 381606 kubeadm.go:393] duration metric: took 49.533039ms to StartCluster
I0416 00:48:01.458199 381606 settings.go:142] acquiring lock: {Name:mkad41a04993d6fe82f2e16230c6052d1c68b809 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:48:01.458256 381606 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18647-2210/kubeconfig
I0416 00:48:01.459621 381606 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-2210/kubeconfig: {Name:mk2a4b2f2d98970b43b7e481fd26cc76bda92838 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:48:01.459852 381606 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.29.3 ContainerRuntime:docker ControlPlane:true Worker:true}
I0416 00:48:01.462416 381606 out.go:177] * Verifying Kubernetes components...
I0416 00:48:01.460143 381606 config.go:182] Loaded profile config "embed-certs-534050": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.29.3
I0416 00:48:01.460165 381606 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0416 00:48:01.464416 381606 addons.go:69] Setting storage-provisioner=true in profile "embed-certs-534050"
I0416 00:48:01.464466 381606 addons.go:234] Setting addon storage-provisioner=true in "embed-certs-534050"
W0416 00:48:01.464478 381606 addons.go:243] addon storage-provisioner should already be in state true
I0416 00:48:01.464506 381606 host.go:66] Checking if "embed-certs-534050" exists ...
I0416 00:48:01.464548 381606 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:48:01.464670 381606 addons.go:69] Setting dashboard=true in profile "embed-certs-534050"
I0416 00:48:01.464710 381606 addons.go:234] Setting addon dashboard=true in "embed-certs-534050"
W0416 00:48:01.464732 381606 addons.go:243] addon dashboard should already be in state true
I0416 00:48:01.464789 381606 host.go:66] Checking if "embed-certs-534050" exists ...
I0416 00:48:01.464966 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:48:01.465286 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:48:01.467297 381606 addons.go:69] Setting default-storageclass=true in profile "embed-certs-534050"
I0416 00:48:01.467341 381606 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "embed-certs-534050"
I0416 00:48:01.467582 381606 addons.go:69] Setting metrics-server=true in profile "embed-certs-534050"
I0416 00:48:01.467613 381606 addons.go:234] Setting addon metrics-server=true in "embed-certs-534050"
W0416 00:48:01.467620 381606 addons.go:243] addon metrics-server should already be in state true
I0416 00:48:01.467641 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:48:01.467649 381606 host.go:66] Checking if "embed-certs-534050" exists ...
I0416 00:48:01.468021 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:48:01.497781 381606 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0416 00:48:01.500318 381606 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:48:01.500337 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0416 00:48:01.500402 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:48:01.518514 381606 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0416 00:48:01.522460 381606 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0416 00:48:01.538938 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0416 00:48:01.538960 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0416 00:48:01.538892 381606 addons.go:234] Setting addon default-storageclass=true in "embed-certs-534050"
W0416 00:48:01.539021 381606 addons.go:243] addon default-storageclass should already be in state true
I0416 00:48:01.539053 381606 host.go:66] Checking if "embed-certs-534050" exists ...
I0416 00:48:01.539128 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:48:01.539576 381606 cli_runner.go:164] Run: docker container inspect embed-certs-534050 --format={{.State.Status}}
I0416 00:48:01.554141 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:48:01.585309 381606 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0416 00:48:01.588778 381606 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0416 00:48:01.588803 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0416 00:48:01.588867 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:48:01.584948 381606 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0416 00:48:01.590941 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0416 00:48:01.591011 381606 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-534050
I0416 00:48:01.593110 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:48:01.613682 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:48:01.634951 381606 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33147 SSHKeyPath:/home/jenkins/minikube-integration/18647-2210/.minikube/machines/embed-certs-534050/id_rsa Username:docker}
I0416 00:48:01.683465 381606 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0416 00:48:01.764147 381606 node_ready.go:35] waiting up to 6m0s for node "embed-certs-534050" to be "Ready" ...
I0416 00:48:01.843616 381606 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0416 00:48:01.929818 381606 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0416 00:48:01.929846 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0416 00:48:02.008552 381606 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0416 00:48:02.015677 381606 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0416 00:48:02.015751 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0416 00:48:02.112166 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0416 00:48:02.112193 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0416 00:48:03.055439 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:05.549612 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:02.189623 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0416 00:48:02.189650 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0416 00:48:02.214821 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0416 00:48:02.214848 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0416 00:48:02.236824 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0416 00:48:02.236849 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0416 00:48:02.307093 381606 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:48:02.307120 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0416 00:48:02.380986 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0416 00:48:02.381012 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0416 00:48:02.624805 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0416 00:48:02.624833 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0416 00:48:02.635417 381606 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0416 00:48:02.702812 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0416 00:48:02.702848 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0416 00:48:02.928190 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0416 00:48:02.928218 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0416 00:48:02.999353 381606 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0416 00:48:02.999394 381606 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0416 00:48:03.086567 381606 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0416 00:48:06.865089 381606 node_ready.go:49] node "embed-certs-534050" has status "Ready":"True"
I0416 00:48:06.865124 381606 node_ready.go:38] duration metric: took 5.100926585s for node "embed-certs-534050" to be "Ready" ...
I0416 00:48:06.865135 381606 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0416 00:48:06.945474 381606 pod_ready.go:78] waiting up to 6m0s for pod "coredns-76f75df574-t2nr8" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.010637 381606 pod_ready.go:92] pod "coredns-76f75df574-t2nr8" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.010676 381606 pod_ready.go:81] duration metric: took 65.160973ms for pod "coredns-76f75df574-t2nr8" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.010689 381606 pod_ready.go:78] waiting up to 6m0s for pod "etcd-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.045010 381606 pod_ready.go:92] pod "etcd-embed-certs-534050" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.045038 381606 pod_ready.go:81] duration metric: took 34.341314ms for pod "etcd-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.045049 381606 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.068197 381606 pod_ready.go:92] pod "kube-apiserver-embed-certs-534050" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.068224 381606 pod_ready.go:81] duration metric: took 23.168241ms for pod "kube-apiserver-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.068244 381606 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.081222 381606 pod_ready.go:92] pod "kube-controller-manager-embed-certs-534050" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.081258 381606 pod_ready.go:81] duration metric: took 13.005681ms for pod "kube-controller-manager-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.081272 381606 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-l8zwp" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.095266 381606 pod_ready.go:92] pod "kube-proxy-l8zwp" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.095303 381606 pod_ready.go:81] duration metric: took 14.014488ms for pod "kube-proxy-l8zwp" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.095319 381606 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:08.050814 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:10.051128 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:07.474200 381606 pod_ready.go:92] pod "kube-scheduler-embed-certs-534050" in "kube-system" namespace has status "Ready":"True"
I0416 00:48:07.474227 381606 pod_ready.go:81] duration metric: took 378.899879ms for pod "kube-scheduler-embed-certs-534050" in "kube-system" namespace to be "Ready" ...
I0416 00:48:07.474239 381606 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace to be "Ready" ...
I0416 00:48:09.496681 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:10.821894 381606 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.978238145s)
I0416 00:48:10.821994 381606 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (8.813416212s)
W0416 00:48:10.822018 381606 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0416 00:48:10.822039 381606 retry.go:31] will retry after 176.459314ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0416 00:48:10.822120 381606 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (8.1866774s)
I0416 00:48:10.822139 381606 addons.go:470] Verifying addon metrics-server=true in "embed-certs-534050"
I0416 00:48:10.965393 381606 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.878765967s)
I0416 00:48:10.967898 381606 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p embed-certs-534050 addons enable metrics-server
I0416 00:48:10.999120 381606 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.29.3/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0416 00:48:11.258847 381606 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0416 00:48:11.261040 381606 addons.go:505] duration metric: took 9.800868584s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0416 00:48:11.981208 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:12.550061 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:15.053238 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:14.480700 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:16.481072 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:17.553163 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:20.050973 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:18.982184 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:20.982643 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:22.053952 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:24.549677 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:23.480851 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:25.481664 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:26.550323 369017 pod_ready.go:102] pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:27.050917 369017 pod_ready.go:81] duration metric: took 4m0.007077036s for pod "metrics-server-9975d5f86-8k8tv" in "kube-system" namespace to be "Ready" ...
E0416 00:48:27.050943 369017 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0416 00:48:27.050953 369017 pod_ready.go:38] duration metric: took 5m26.131424924s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0416 00:48:27.050972 369017 api_server.go:52] waiting for apiserver process to appear ...
I0416 00:48:27.051054 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0416 00:48:27.073537 369017 logs.go:276] 2 containers: [a7d7845d2402 b8ea3fa2ab02]
I0416 00:48:27.073627 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0416 00:48:27.092597 369017 logs.go:276] 2 containers: [33107d331e0b fd5230a8d74b]
I0416 00:48:27.092711 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0416 00:48:27.123060 369017 logs.go:276] 2 containers: [65e7340af5ef 697870ff99a4]
I0416 00:48:27.123158 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0416 00:48:27.141834 369017 logs.go:276] 2 containers: [2d7d1b9e8353 7b437d823755]
I0416 00:48:27.141920 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0416 00:48:27.160491 369017 logs.go:276] 2 containers: [fa54eb276fa9 bf3ceb2acadb]
I0416 00:48:27.160587 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0416 00:48:27.180907 369017 logs.go:276] 2 containers: [b3c3c455ea1c 4cc3ed1cf27e]
I0416 00:48:27.180992 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0416 00:48:27.198507 369017 logs.go:276] 0 containers: []
W0416 00:48:27.198530 369017 logs.go:278] No container was found matching "kindnet"
I0416 00:48:27.198587 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0416 00:48:27.216044 369017 logs.go:276] 1 containers: [c311fb93e11b]
I0416 00:48:27.216241 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0416 00:48:27.232689 369017 logs.go:276] 2 containers: [c5eb3f5fa95a d8e8f85e95c4]
I0416 00:48:27.232722 369017 logs.go:123] Gathering logs for kubernetes-dashboard [c311fb93e11b] ...
I0416 00:48:27.232735 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c311fb93e11b"
I0416 00:48:27.254742 369017 logs.go:123] Gathering logs for kube-apiserver [a7d7845d2402] ...
I0416 00:48:27.254816 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 a7d7845d2402"
I0416 00:48:27.299302 369017 logs.go:123] Gathering logs for coredns [65e7340af5ef] ...
I0416 00:48:27.299340 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 65e7340af5ef"
I0416 00:48:27.327427 369017 logs.go:123] Gathering logs for kube-scheduler [7b437d823755] ...
I0416 00:48:27.327461 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b437d823755"
I0416 00:48:27.354603 369017 logs.go:123] Gathering logs for kube-proxy [bf3ceb2acadb] ...
I0416 00:48:27.354636 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 bf3ceb2acadb"
I0416 00:48:27.377974 369017 logs.go:123] Gathering logs for storage-provisioner [c5eb3f5fa95a] ...
I0416 00:48:27.378013 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c5eb3f5fa95a"
I0416 00:48:27.400538 369017 logs.go:123] Gathering logs for storage-provisioner [d8e8f85e95c4] ...
I0416 00:48:27.400566 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 d8e8f85e95c4"
I0416 00:48:27.421475 369017 logs.go:123] Gathering logs for container status ...
I0416 00:48:27.421505 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0416 00:48:27.501109 369017 logs.go:123] Gathering logs for kubelet ...
I0416 00:48:27.501142 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0416 00:48:27.578790 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:03 old-k8s-version-014065 kubelet[1223]: E0416 00:43:03.441058 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.580653 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:04 old-k8s-version-014065 kubelet[1223]: E0416 00:43:04.639437 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.583270 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:15 old-k8s-version-014065 kubelet[1223]: E0416 00:43:15.964300 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.592833 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:20 old-k8s-version-014065 kubelet[1223]: E0416 00:43:20.969599 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.593059 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:21 old-k8s-version-014065 kubelet[1223]: E0416 00:43:21.832458 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.594338 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:26 old-k8s-version-014065 kubelet[1223]: E0416 00:43:26.931292 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.597710 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.428287 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.598545 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.976063 1223 pod_workers.go:191] Error syncing pod cef2692c-ceee-4c9c-892a-75dcaae5ab8a ("storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"
W0416 00:48:27.600747 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:38 old-k8s-version-014065 kubelet[1223]: E0416 00:43:38.955013 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.601295 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:46 old-k8s-version-014065 kubelet[1223]: E0416 00:43:46.931311 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.601614 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:53 old-k8s-version-014065 kubelet[1223]: E0416 00:43:53.963232 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.604001 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:02 old-k8s-version-014065 kubelet[1223]: E0416 00:44:02.430723 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.604206 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:07 old-k8s-version-014065 kubelet[1223]: E0416 00:44:07.936331 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.604402 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:14 old-k8s-version-014065 kubelet[1223]: E0416 00:44:14.931692 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.606653 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:20 old-k8s-version-014065 kubelet[1223]: E0416 00:44:20.968241 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.606863 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:25 old-k8s-version-014065 kubelet[1223]: E0416 00:44:25.952535 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607048 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:32 old-k8s-version-014065 kubelet[1223]: E0416 00:44:32.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607278 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:40 old-k8s-version-014065 kubelet[1223]: E0416 00:44:40.931086 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.607466 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:46 old-k8s-version-014065 kubelet[1223]: E0416 00:44:46.931443 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.609801 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:54 old-k8s-version-014065 kubelet[1223]: E0416 00:44:54.366332 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.610002 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:01 old-k8s-version-014065 kubelet[1223]: E0416 00:45:01.931372 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610273 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:08 old-k8s-version-014065 kubelet[1223]: E0416 00:45:08.933619 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610493 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:12 old-k8s-version-014065 kubelet[1223]: E0416 00:45:12.931273 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610681 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.931035 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.610917 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.953472 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.611117 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:35 old-k8s-version-014065 kubelet[1223]: E0416 00:45:35.932086 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.611363 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:38 old-k8s-version-014065 kubelet[1223]: E0416 00:45:38.937563 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.613506 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:49 old-k8s-version-014065 kubelet[1223]: E0416 00:45:49.959737 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:27.613707 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:50 old-k8s-version-014065 kubelet[1223]: E0416 00:45:50.931084 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.613892 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:02 old-k8s-version-014065 kubelet[1223]: E0416 00:46:02.934939 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.614087 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:04 old-k8s-version-014065 kubelet[1223]: E0416 00:46:04.931137 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.614275 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:14 old-k8s-version-014065 kubelet[1223]: E0416 00:46:14.942668 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.616488 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:16 old-k8s-version-014065 kubelet[1223]: E0416 00:46:16.403347 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:27.616677 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:27 old-k8s-version-014065 kubelet[1223]: E0416 00:46:27.930989 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.616873 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:30 old-k8s-version-014065 kubelet[1223]: E0416 00:46:30.931502 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617084 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:40 old-k8s-version-014065 kubelet[1223]: E0416 00:46:40.931581 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617280 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:45 old-k8s-version-014065 kubelet[1223]: E0416 00:46:45.971319 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617464 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:53 old-k8s-version-014065 kubelet[1223]: E0416 00:46:53.934295 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617670 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:56 old-k8s-version-014065 kubelet[1223]: E0416 00:46:56.930865 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.617855 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:05 old-k8s-version-014065 kubelet[1223]: E0416 00:47:05.931075 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618050 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:10 old-k8s-version-014065 kubelet[1223]: E0416 00:47:10.931300 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618235 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:20 old-k8s-version-014065 kubelet[1223]: E0416 00:47:20.932206 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618431 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:25 old-k8s-version-014065 kubelet[1223]: E0416 00:47:25.931180 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618616 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:31 old-k8s-version-014065 kubelet[1223]: E0416 00:47:31.931916 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.618840 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:37 old-k8s-version-014065 kubelet[1223]: E0416 00:47:37.931161 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619026 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:44 old-k8s-version-014065 kubelet[1223]: E0416 00:47:44.931376 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619351 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:48 old-k8s-version-014065 kubelet[1223]: E0416 00:47:48.937314 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619548 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619745 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.619929 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.620123 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:27.620308 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0416 00:48:27.620319 369017 logs.go:123] Gathering logs for kube-apiserver [b8ea3fa2ab02] ...
I0416 00:48:27.620334 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b8ea3fa2ab02"
I0416 00:48:27.698206 369017 logs.go:123] Gathering logs for coredns [697870ff99a4] ...
I0416 00:48:27.698250 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 697870ff99a4"
I0416 00:48:27.720130 369017 logs.go:123] Gathering logs for kube-scheduler [2d7d1b9e8353] ...
I0416 00:48:27.720164 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 2d7d1b9e8353"
I0416 00:48:27.741457 369017 logs.go:123] Gathering logs for dmesg ...
I0416 00:48:27.741485 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0416 00:48:27.764244 369017 logs.go:123] Gathering logs for describe nodes ...
I0416 00:48:27.764279 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0416 00:48:27.930626 369017 logs.go:123] Gathering logs for kube-proxy [fa54eb276fa9] ...
I0416 00:48:27.930654 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fa54eb276fa9"
I0416 00:48:27.962973 369017 logs.go:123] Gathering logs for Docker ...
I0416 00:48:27.963046 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0416 00:48:27.999906 369017 logs.go:123] Gathering logs for etcd [33107d331e0b] ...
I0416 00:48:27.999940 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 33107d331e0b"
I0416 00:48:28.028776 369017 logs.go:123] Gathering logs for etcd [fd5230a8d74b] ...
I0416 00:48:28.028817 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fd5230a8d74b"
I0416 00:48:28.064939 369017 logs.go:123] Gathering logs for kube-controller-manager [b3c3c455ea1c] ...
I0416 00:48:28.064970 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b3c3c455ea1c"
I0416 00:48:28.117348 369017 logs.go:123] Gathering logs for kube-controller-manager [4cc3ed1cf27e] ...
I0416 00:48:28.117379 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4cc3ed1cf27e"
I0416 00:48:28.175750 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:28.175783 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0416 00:48:28.175846 369017 out.go:239] X Problems detected in kubelet:
W0416 00:48:28.175861 369017 out.go:239] Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175875 369017 out.go:239] Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175885 369017 out.go:239] Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175896 369017 out.go:239] Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:28.175904 369017 out.go:239] Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0416 00:48:28.175912 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:28.175920 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:48:27.981703 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:29.983323 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:32.480320 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:34.481646 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:36.983474 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:38.176694 369017 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0416 00:48:38.191932 369017 api_server.go:72] duration metric: took 5m49.421692349s to wait for apiserver process to appear ...
I0416 00:48:38.191963 369017 api_server.go:88] waiting for apiserver healthz status ...
I0416 00:48:38.192051 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0416 00:48:38.209623 369017 logs.go:276] 2 containers: [a7d7845d2402 b8ea3fa2ab02]
I0416 00:48:38.209701 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0416 00:48:38.228837 369017 logs.go:276] 2 containers: [33107d331e0b fd5230a8d74b]
I0416 00:48:38.228918 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0416 00:48:38.245852 369017 logs.go:276] 2 containers: [65e7340af5ef 697870ff99a4]
I0416 00:48:38.245943 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0416 00:48:38.268054 369017 logs.go:276] 2 containers: [2d7d1b9e8353 7b437d823755]
I0416 00:48:38.268136 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0416 00:48:38.292495 369017 logs.go:276] 2 containers: [fa54eb276fa9 bf3ceb2acadb]
I0416 00:48:38.292571 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0416 00:48:38.314076 369017 logs.go:276] 2 containers: [b3c3c455ea1c 4cc3ed1cf27e]
I0416 00:48:38.314160 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0416 00:48:38.330761 369017 logs.go:276] 0 containers: []
W0416 00:48:38.330781 369017 logs.go:278] No container was found matching "kindnet"
I0416 00:48:38.330834 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0416 00:48:38.349329 369017 logs.go:276] 2 containers: [c5eb3f5fa95a d8e8f85e95c4]
I0416 00:48:38.349479 369017 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0416 00:48:38.366973 369017 logs.go:276] 1 containers: [c311fb93e11b]
I0416 00:48:38.367003 369017 logs.go:123] Gathering logs for kube-apiserver [b8ea3fa2ab02] ...
I0416 00:48:38.367015 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b8ea3fa2ab02"
I0416 00:48:38.422460 369017 logs.go:123] Gathering logs for etcd [33107d331e0b] ...
I0416 00:48:38.422494 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 33107d331e0b"
I0416 00:48:38.459188 369017 logs.go:123] Gathering logs for kube-scheduler [2d7d1b9e8353] ...
I0416 00:48:38.459287 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 2d7d1b9e8353"
I0416 00:48:38.488818 369017 logs.go:123] Gathering logs for kube-controller-manager [b3c3c455ea1c] ...
I0416 00:48:38.488856 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 b3c3c455ea1c"
I0416 00:48:38.528135 369017 logs.go:123] Gathering logs for kubelet ...
I0416 00:48:38.528170 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0416 00:48:38.584942 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:03 old-k8s-version-014065 kubelet[1223]: E0416 00:43:03.441058 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.586673 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:04 old-k8s-version-014065 kubelet[1223]: E0416 00:43:04.639437 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.589065 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:15 old-k8s-version-014065 kubelet[1223]: E0416 00:43:15.964300 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.593156 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:20 old-k8s-version-014065 kubelet[1223]: E0416 00:43:20.969599 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.593356 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:21 old-k8s-version-014065 kubelet[1223]: E0416 00:43:21.832458 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.594043 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:26 old-k8s-version-014065 kubelet[1223]: E0416 00:43:26.931292 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.596270 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.428287 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.597041 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:33 old-k8s-version-014065 kubelet[1223]: E0416 00:43:33.976063 1223 pod_workers.go:191] Error syncing pod cef2692c-ceee-4c9c-892a-75dcaae5ab8a ("storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(cef2692c-ceee-4c9c-892a-75dcaae5ab8a)"
W0416 00:48:38.599100 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:38 old-k8s-version-014065 kubelet[1223]: E0416 00:43:38.955013 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.599665 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:46 old-k8s-version-014065 kubelet[1223]: E0416 00:43:46.931311 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.599979 369017 logs.go:138] Found kubelet problem: Apr 16 00:43:53 old-k8s-version-014065 kubelet[1223]: E0416 00:43:53.963232 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.602177 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:02 old-k8s-version-014065 kubelet[1223]: E0416 00:44:02.430723 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.602364 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:07 old-k8s-version-014065 kubelet[1223]: E0416 00:44:07.936331 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.602559 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:14 old-k8s-version-014065 kubelet[1223]: E0416 00:44:14.931692 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.604596 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:20 old-k8s-version-014065 kubelet[1223]: E0416 00:44:20.968241 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.604793 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:25 old-k8s-version-014065 kubelet[1223]: E0416 00:44:25.952535 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.604977 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:32 old-k8s-version-014065 kubelet[1223]: E0416 00:44:32.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.605172 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:40 old-k8s-version-014065 kubelet[1223]: E0416 00:44:40.931086 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.605356 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:46 old-k8s-version-014065 kubelet[1223]: E0416 00:44:46.931443 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.607588 369017 logs.go:138] Found kubelet problem: Apr 16 00:44:54 old-k8s-version-014065 kubelet[1223]: E0416 00:44:54.366332 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.607775 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:01 old-k8s-version-014065 kubelet[1223]: E0416 00:45:01.931372 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.607984 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:08 old-k8s-version-014065 kubelet[1223]: E0416 00:45:08.933619 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608167 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:12 old-k8s-version-014065 kubelet[1223]: E0416 00:45:12.931273 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608349 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.931035 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608545 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:23 old-k8s-version-014065 kubelet[1223]: E0416 00:45:23.953472 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608728 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:35 old-k8s-version-014065 kubelet[1223]: E0416 00:45:35.932086 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.608930 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:38 old-k8s-version-014065 kubelet[1223]: E0416 00:45:38.937563 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.610973 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:49 old-k8s-version-014065 kubelet[1223]: E0416 00:45:49.959737 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0416 00:48:38.611166 369017 logs.go:138] Found kubelet problem: Apr 16 00:45:50 old-k8s-version-014065 kubelet[1223]: E0416 00:45:50.931084 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611382 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:02 old-k8s-version-014065 kubelet[1223]: E0416 00:46:02.934939 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611578 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:04 old-k8s-version-014065 kubelet[1223]: E0416 00:46:04.931137 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.611762 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:14 old-k8s-version-014065 kubelet[1223]: E0416 00:46:14.942668 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.613954 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:16 old-k8s-version-014065 kubelet[1223]: E0416 00:46:16.403347 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0416 00:48:38.614138 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:27 old-k8s-version-014065 kubelet[1223]: E0416 00:46:27.930989 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614333 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:30 old-k8s-version-014065 kubelet[1223]: E0416 00:46:30.931502 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614519 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:40 old-k8s-version-014065 kubelet[1223]: E0416 00:46:40.931581 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614712 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:45 old-k8s-version-014065 kubelet[1223]: E0416 00:46:45.971319 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.614911 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:53 old-k8s-version-014065 kubelet[1223]: E0416 00:46:53.934295 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615106 369017 logs.go:138] Found kubelet problem: Apr 16 00:46:56 old-k8s-version-014065 kubelet[1223]: E0416 00:46:56.930865 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615296 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:05 old-k8s-version-014065 kubelet[1223]: E0416 00:47:05.931075 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615490 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:10 old-k8s-version-014065 kubelet[1223]: E0416 00:47:10.931300 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615673 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:20 old-k8s-version-014065 kubelet[1223]: E0416 00:47:20.932206 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.615866 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:25 old-k8s-version-014065 kubelet[1223]: E0416 00:47:25.931180 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616049 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:31 old-k8s-version-014065 kubelet[1223]: E0416 00:47:31.931916 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616243 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:37 old-k8s-version-014065 kubelet[1223]: E0416 00:47:37.931161 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616448 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:44 old-k8s-version-014065 kubelet[1223]: E0416 00:47:44.931376 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616644 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:48 old-k8s-version-014065 kubelet[1223]: E0416 00:47:48.937314 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.616827 369017 logs.go:138] Found kubelet problem: Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617022 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617204 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617401 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617586 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:38.617782 369017 logs.go:138] Found kubelet problem: Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0416 00:48:38.617792 369017 logs.go:123] Gathering logs for kube-apiserver [a7d7845d2402] ...
I0416 00:48:38.617805 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 a7d7845d2402"
I0416 00:48:38.659232 369017 logs.go:123] Gathering logs for coredns [65e7340af5ef] ...
I0416 00:48:38.659348 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 65e7340af5ef"
I0416 00:48:38.688264 369017 logs.go:123] Gathering logs for kube-proxy [fa54eb276fa9] ...
I0416 00:48:38.688293 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fa54eb276fa9"
I0416 00:48:38.708927 369017 logs.go:123] Gathering logs for kube-proxy [bf3ceb2acadb] ...
I0416 00:48:38.708955 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 bf3ceb2acadb"
I0416 00:48:38.729344 369017 logs.go:123] Gathering logs for kube-controller-manager [4cc3ed1cf27e] ...
I0416 00:48:38.729374 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4cc3ed1cf27e"
I0416 00:48:38.771163 369017 logs.go:123] Gathering logs for Docker ...
I0416 00:48:38.771228 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0416 00:48:38.802002 369017 logs.go:123] Gathering logs for dmesg ...
I0416 00:48:38.802041 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0416 00:48:38.822471 369017 logs.go:123] Gathering logs for describe nodes ...
I0416 00:48:38.822500 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0416 00:48:38.978993 369017 logs.go:123] Gathering logs for container status ...
I0416 00:48:38.979023 369017 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0416 00:48:39.074662 369017 logs.go:123] Gathering logs for coredns [697870ff99a4] ...
I0416 00:48:39.074703 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 697870ff99a4"
I0416 00:48:39.100511 369017 logs.go:123] Gathering logs for storage-provisioner [d8e8f85e95c4] ...
I0416 00:48:39.100545 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 d8e8f85e95c4"
I0416 00:48:39.122784 369017 logs.go:123] Gathering logs for storage-provisioner [c5eb3f5fa95a] ...
I0416 00:48:39.122813 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c5eb3f5fa95a"
I0416 00:48:39.144632 369017 logs.go:123] Gathering logs for kubernetes-dashboard [c311fb93e11b] ...
I0416 00:48:39.144662 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 c311fb93e11b"
I0416 00:48:39.166870 369017 logs.go:123] Gathering logs for etcd [fd5230a8d74b] ...
I0416 00:48:39.166901 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 fd5230a8d74b"
I0416 00:48:39.190433 369017 logs.go:123] Gathering logs for kube-scheduler [7b437d823755] ...
I0416 00:48:39.190464 369017 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b437d823755"
I0416 00:48:39.213048 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:39.213073 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0416 00:48:39.213162 369017 out.go:239] X Problems detected in kubelet:
W0416 00:48:39.213174 369017 out.go:239] Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213185 369017 out.go:239] Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213207 369017 out.go:239] Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213217 369017 out.go:239] Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0416 00:48:39.213223 369017 out.go:239] Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0416 00:48:39.213235 369017 out.go:304] Setting ErrFile to fd 2...
I0416 00:48:39.213242 369017 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:48:38.985116 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:41.482785 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:43.981777 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:45.981918 381606 pod_ready.go:102] pod "metrics-server-57f55c9bc5-4bqxl" in "kube-system" namespace has status "Ready":"False"
I0416 00:48:49.213969 369017 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0416 00:48:49.227100 369017 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0416 00:48:49.230281 369017 out.go:177]
W0416 00:48:49.232609 369017 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0416 00:48:49.232667 369017 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0416 00:48:49.232693 369017 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0416 00:48:49.232702 369017 out.go:239] *
W0416 00:48:49.234077 369017 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0416 00:48:49.236416 369017 out.go:177]
==> Docker <==
Apr 16 00:48:27 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:27 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:27 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:27 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:27 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:27 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:28 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:28 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:28 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:28 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:28 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:28 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:28 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:28 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:38 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:38 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: 2024/04/16 00:48:39 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: time="2024-04-16T00:48:39.956667271Z" level=warning msg="Error getting v2 registry: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=fc7bb4fc2dbd3e7a traceID=50d125c714d09db8e72f91157c2cb944
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: time="2024-04-16T00:48:39.956725624Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=fc7bb4fc2dbd3e7a traceID=50d125c714d09db8e72f91157c2cb944
Apr 16 00:48:39 old-k8s-version-014065 dockerd[979]: time="2024-04-16T00:48:39.960407120Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=fc7bb4fc2dbd3e7a traceID=50d125c714d09db8e72f91157c2cb944
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
c5eb3f5fa95a3 ba04bb24b9575 5 minutes ago Running storage-provisioner 3 0623b27a16861 storage-provisioner
c311fb93e11b4 kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93 5 minutes ago Running kubernetes-dashboard 0 d3e602322d94e kubernetes-dashboard-cd95d586-gh7qv
65e7340af5efc db91994f4ee8f 5 minutes ago Running coredns 1 a2fe98efc4317 coredns-74ff55c5b-ftt5t
d8e8f85e95c4e ba04bb24b9575 5 minutes ago Exited storage-provisioner 2 0623b27a16861 storage-provisioner
0575ad4292326 1611cd07b61d5 5 minutes ago Running busybox 1 fabd6d1c23420 busybox
fa54eb276fa99 25a5233254979 5 minutes ago Running kube-proxy 1 e0384a5b397ea kube-proxy-2ltgk
33107d331e0b4 05b738aa1bc63 5 minutes ago Running etcd 1 c6b4fd46b8277 etcd-old-k8s-version-014065
a7d7845d2402e 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 8b758f0778c42 kube-apiserver-old-k8s-version-014065
2d7d1b9e83535 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 7a01f79623eef kube-scheduler-old-k8s-version-014065
b3c3c455ea1c0 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 629f506fab1f4 kube-controller-manager-old-k8s-version-014065
5ce6cb9441727 gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e 6 minutes ago Exited busybox 0 82cf1fd86125c busybox
697870ff99a46 db91994f4ee8f 8 minutes ago Exited coredns 0 5e996201801f3 coredns-74ff55c5b-ftt5t
bf3ceb2acadb3 25a5233254979 8 minutes ago Exited kube-proxy 0 7cf84019e80dd kube-proxy-2ltgk
7b437d8237553 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 9d5b39c918112 kube-scheduler-old-k8s-version-014065
4cc3ed1cf27ed 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 b4f27bb094b7c kube-controller-manager-old-k8s-version-014065
b8ea3fa2ab024 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 67fe14533f1ad kube-apiserver-old-k8s-version-014065
fd5230a8d74bd 05b738aa1bc63 8 minutes ago Exited etcd 0 5d7b40ae19a43 etcd-old-k8s-version-014065
==> coredns [65e7340af5ef] <==
[INFO] plugin/ready: Still waiting on: "kubernetes"
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:50154 - 53633 "HINFO IN 8120355963274842863.2253897167941298015. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.031784472s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0416 00:43:33.816498 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:43:03.813011041 +0000 UTC m=+0.081684558) (total time: 30.001969867s):
Trace[2019727887]: [30.001969867s] [30.001969867s] END
E0416 00:43:33.816550 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0416 00:43:33.816506 1 trace.go:116] Trace[1427131847]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:43:03.812935629 +0000 UTC m=+0.081609155) (total time: 30.002064445s):
Trace[1427131847]: [30.002064445s] [30.002064445s] END
E0416 00:43:33.816579 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0416 00:43:33.816781 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:43:03.814435321 +0000 UTC m=+0.083108830) (total time: 30.002328891s):
Trace[911902081]: [30.002328891s] [30.002328891s] END
E0416 00:43:33.816794 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [697870ff99a4] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] Reloading
[INFO] plugin/health: Going into lameduck mode for 5s
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
[INFO] Reloading complete
[INFO] 127.0.0.1:51946 - 12757 "HINFO IN 3475134916925666260.1098169330899469510. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.060809881s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
I0416 00:41:11.971146 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:40:41.97055784 +0000 UTC m=+0.054831162) (total time: 30.000489285s):
Trace[2019727887]: [30.000489285s] [30.000489285s] END
E0416 00:41:11.971188 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0416 00:41:11.971643 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:40:41.971144231 +0000 UTC m=+0.055417544) (total time: 30.000480096s):
Trace[939984059]: [30.000480096s] [30.000480096s] END
E0416 00:41:11.971658 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0416 00:41:11.971757 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-04-16 00:40:41.971397706 +0000 UTC m=+0.055671028) (total time: 30.000333448s):
Trace[911902081]: [30.000333448s] [30.000333448s] END
E0416 00:41:11.971766 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> describe nodes <==
Name: old-k8s-version-014065
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-014065
kubernetes.io/os=linux
minikube.k8s.io/commit=e98a202b00bb48daab8bbee2f0c7bcf1556f8388
minikube.k8s.io/name=old-k8s-version-014065
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_04_16T00_40_20_0700
minikube.k8s.io/version=v1.33.0-beta.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 16 Apr 2024 00:40:17 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-014065
AcquireTime: <unset>
RenewTime: Tue, 16 Apr 2024 00:48:43 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 16 Apr 2024 00:43:51 +0000 Tue, 16 Apr 2024 00:40:09 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 16 Apr 2024 00:43:51 +0000 Tue, 16 Apr 2024 00:40:09 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 16 Apr 2024 00:43:51 +0000 Tue, 16 Apr 2024 00:40:09 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 16 Apr 2024 00:43:51 +0000 Tue, 16 Apr 2024 00:40:35 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-014065
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
System Info:
Machine ID: 00e766457fe04ed39a5327d64fb0cf5f
System UUID: 132da16a-063e-4e5b-9748-6b3cf0c499e0
Boot ID: ed177bf7-a11f-466b-8935-e2b8479e05ab
Kernel Version: 5.15.0-1057-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: docker://26.0.1
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (11 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m32s
kube-system coredns-74ff55c5b-ftt5t 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m13s
kube-system etcd-old-k8s-version-014065 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m24s
kube-system kube-apiserver-old-k8s-version-014065 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m24s
kube-system kube-controller-manager-old-k8s-version-014065 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m24s
kube-system kube-proxy-2ltgk 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m13s
kube-system kube-scheduler-old-k8s-version-014065 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m24s
kube-system metrics-server-9975d5f86-8k8tv 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m21s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m11s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-6qzsp 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m31s
kubernetes-dashboard kubernetes-dashboard-cd95d586-gh7qv 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m31s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 850m (42%!)(MISSING) 0 (0%!)(MISSING)
memory 370Mi (4%!)(MISSING) 170Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m42s (x5 over 8m42s) kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m42s (x5 over 8m42s) kubelet Node old-k8s-version-014065 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m42s (x4 over 8m42s) kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientPID
Normal Starting 8m26s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m26s kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m26s kubelet Node old-k8s-version-014065 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m26s kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m25s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m15s kubelet Node old-k8s-version-014065 status is now: NodeReady
Normal Starting 8m9s kube-proxy Starting kube-proxy.
Normal Starting 5m59s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m58s (x8 over 5m59s) kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m58s (x8 over 5m59s) kubelet Node old-k8s-version-014065 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m58s (x7 over 5m59s) kubelet Node old-k8s-version-014065 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m58s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m47s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.000919] FS-Cache: N-cookie d=000000003a70b87a{9p.inode} n=0000000082e46d16
[ +0.001014] FS-Cache: N-key=[8] '9a6ced0000000000'
[ +0.002893] FS-Cache: Duplicate cookie detected
[ +0.000699] FS-Cache: O-cookie c=00000006 [p=00000003 fl=226 nc=0 na=1]
[ +0.000945] FS-Cache: O-cookie d=000000003a70b87a{9p.inode} n=0000000065060579
[ +0.001199] FS-Cache: O-key=[8] '9a6ced0000000000'
[ +0.000702] FS-Cache: N-cookie c=0000000d [p=00000003 fl=2 nc=0 na=1]
[ +0.000911] FS-Cache: N-cookie d=000000003a70b87a{9p.inode} n=00000000e3ab24ac
[ +0.001035] FS-Cache: N-key=[8] '9a6ced0000000000'
[ +2.693241] FS-Cache: Duplicate cookie detected
[ +0.000715] FS-Cache: O-cookie c=00000004 [p=00000003 fl=226 nc=0 na=1]
[ +0.001012] FS-Cache: O-cookie d=000000003a70b87a{9p.inode} n=000000008f112200
[ +0.001044] FS-Cache: O-key=[8] '996ced0000000000'
[ +0.000769] FS-Cache: N-cookie c=0000000f [p=00000003 fl=2 nc=0 na=1]
[ +0.000950] FS-Cache: N-cookie d=000000003a70b87a{9p.inode} n=00000000a69e10ca
[ +0.001055] FS-Cache: N-key=[8] '996ced0000000000'
[ +0.334896] FS-Cache: Duplicate cookie detected
[ +0.000796] FS-Cache: O-cookie c=00000009 [p=00000003 fl=226 nc=0 na=1]
[ +0.000957] FS-Cache: O-cookie d=000000003a70b87a{9p.inode} n=0000000018d51e64
[ +0.001059] FS-Cache: O-key=[8] 'a16ced0000000000'
[ +0.000711] FS-Cache: N-cookie c=00000010 [p=00000003 fl=2 nc=0 na=1]
[ +0.000930] FS-Cache: N-cookie d=000000003a70b87a{9p.inode} n=00000000910ed9b0
[ +0.001068] FS-Cache: N-key=[8] 'a16ced0000000000'
[Apr15 23:52] hrtimer: interrupt took 27400523 ns
[Apr16 00:27] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.
==> etcd [33107d331e0b] <==
2024-04-16 00:44:48.194789 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:44:58.194506 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:08.194432 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:18.194433 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:28.194334 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:38.194279 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:48.194225 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:45:58.194458 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:08.194449 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:18.194433 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:28.194305 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:38.194305 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:48.194457 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:46:58.195130 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:08.194352 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:18.194558 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:28.194364 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:38.194386 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:48.194558 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:47:58.199437 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:48:08.194439 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:48:18.194306 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:48:28.194460 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:48:38.194538 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:48:48.194429 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [fd5230a8d74b] <==
raft2024/04/16 00:40:10 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-04-16 00:40:10.090911 I | etcdserver: setting up the initial cluster version to 3.4
2024-04-16 00:40:10.091617 N | etcdserver/membership: set the initial cluster version to 3.4
2024-04-16 00:40:10.091854 I | etcdserver/api: enabled capabilities for version 3.4
2024-04-16 00:40:10.092025 I | etcdserver: published {Name:old-k8s-version-014065 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-04-16 00:40:10.092300 I | embed: ready to serve client requests
2024-04-16 00:40:10.095626 I | embed: serving client requests on 127.0.0.1:2379
2024-04-16 00:40:10.095896 I | embed: ready to serve client requests
2024-04-16 00:40:10.099891 I | embed: serving client requests on 192.168.76.2:2379
2024-04-16 00:40:29.251021 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:40:29.577533 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:40:39.591649 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:40:49.577449 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:40:59.577640 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:09.577658 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:19.577492 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:29.578987 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:39.577545 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:49.577451 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:41:59.577858 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:42:09.577654 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:42:19.577432 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-04-16 00:42:29.860439 N | pkg/osutil: received terminated signal, shutting down...
WARNING: 2024/04/16 00:42:29 grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
2024-04-16 00:42:29.919156 I | etcdserver: skipped leadership transfer for single voting member cluster
==> kernel <==
00:48:50 up 1:31, 0 users, load average: 3.51, 3.44, 3.76
Linux old-k8s-version-014065 5.15.0-1057-aws #63~20.04.1-Ubuntu SMP Mon Mar 25 10:29:14 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kube-apiserver [a7d7845d2402] <==
I0416 00:45:48.553559 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:45:48.553568 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0416 00:46:04.102434 1 handler_proxy.go:102] no RequestInfo found in the context
E0416 00:46:04.102507 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0416 00:46:04.102547 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0416 00:46:25.880729 1 client.go:360] parsed scheme: "passthrough"
I0416 00:46:25.880783 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:46:25.880793 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0416 00:47:05.660986 1 client.go:360] parsed scheme: "passthrough"
I0416 00:47:05.661175 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:47:05.661197 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0416 00:47:37.369183 1 client.go:360] parsed scheme: "passthrough"
I0416 00:47:37.369234 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:47:37.369242 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0416 00:48:01.912415 1 handler_proxy.go:102] no RequestInfo found in the context
E0416 00:48:01.912500 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0416 00:48:01.912512 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0416 00:48:07.599015 1 client.go:360] parsed scheme: "passthrough"
I0416 00:48:07.599061 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:48:07.599070 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0416 00:48:40.438763 1 client.go:360] parsed scheme: "passthrough"
I0416 00:48:40.438827 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0416 00:48:40.438837 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [b8ea3fa2ab02] <==
I0416 00:42:29.989968 1 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
W0416 00:42:29.990120 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990173 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990213 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990249 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990277 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990305 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990336 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990370 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990406 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990441 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990477 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990517 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990555 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990592 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990629 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990670 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990710 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990743 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990777 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990820 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990915 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.990962 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.991042 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0416 00:42:29.991076 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
==> kube-controller-manager [4cc3ed1cf27e] <==
I0416 00:40:37.046270 1 taint_manager.go:187] Starting NoExecuteTaintManager
I0416 00:40:37.046894 1 event.go:291] "Event occurred" object="old-k8s-version-014065" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-014065 event: Registered Node old-k8s-version-014065 in Controller"
I0416 00:40:37.047831 1 shared_informer.go:247] Caches are synced for namespace
I0416 00:40:37.053395 1 shared_informer.go:247] Caches are synced for PVC protection
I0416 00:40:37.063632 1 shared_informer.go:247] Caches are synced for GC
I0416 00:40:37.077751 1 shared_informer.go:247] Caches are synced for daemon sets
I0416 00:40:37.149839 1 range_allocator.go:373] Set node old-k8s-version-014065 PodCIDR to [10.244.0.0/24]
I0416 00:40:37.177696 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0416 00:40:37.222724 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-h6wrz"
I0416 00:40:37.233388 1 shared_informer.go:247] Caches are synced for resource quota
I0416 00:40:37.245365 1 shared_informer.go:247] Caches are synced for disruption
I0416 00:40:37.245386 1 disruption.go:339] Sending events to api server.
I0416 00:40:37.252563 1 shared_informer.go:247] Caches are synced for ReplicationController
I0416 00:40:37.268838 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-ftt5t"
I0416 00:40:37.269141 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-2ltgk"
I0416 00:40:37.271542 1 shared_informer.go:247] Caches are synced for resource quota
I0416 00:40:37.271812 1 shared_informer.go:247] Caches are synced for attach detach
I0416 00:40:37.434474 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
E0416 00:40:37.468068 1 daemon_controller.go:320] kube-system/kube-proxy failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-proxy", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"54eb5f82-4ba2-4f5e-a608-e252f9a59595", ResourceVersion:"268", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63848824820, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubeadm", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001b43c20), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001b43c40)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.
LabelSelector)(0x4001b43c60), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"kube-proxy", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Gl
usterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0x4001b60cc0), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001b43
c80), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeS
ource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001b43ca0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil),
AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kube-proxy", Image:"k8s.gcr.io/kube-proxy:v1.20.0", Command:[]string{"/usr/local/bin/kube-proxy", "--config=/var/lib/kube-proxy/config.conf", "--hostname-override=$(NODE_NAME)"}, Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001b43ce0)}}, Resources:v1.R
esourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList(nil)}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"kube-proxy", ReadOnly:false, MountPath:"/var/lib/kube-proxy", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x4001a91740), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPo
licy:"Always", TerminationGracePeriodSeconds:(*int64)(0x4001b1b118), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"kube-proxy", DeprecatedServiceAccount:"kube-proxy", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x400002ca80), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), Runtime
ClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x4001b30278)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x4001b1b168)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again
I0416 00:40:37.644609 1 shared_informer.go:247] Caches are synced for garbage collector
I0416 00:40:37.697583 1 shared_informer.go:247] Caches are synced for garbage collector
I0416 00:40:37.697607 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0416 00:40:39.853190 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0416 00:40:39.893587 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-h6wrz"
I0416 00:42:28.456767 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
==> kube-controller-manager [b3c3c455ea1c] <==
W0416 00:44:25.201150 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:44:51.215732 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:44:56.851554 1 request.go:655] Throttling request took 1.048499968s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1beta1?timeout=32s
W0416 00:44:57.702988 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:45:21.717613 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:45:29.353353 1 request.go:655] Throttling request took 1.048239333s, request: GET:https://192.168.76.2:8443/apis/batch/v1?timeout=32s
W0416 00:45:30.209003 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:45:52.219787 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:46:01.873944 1 request.go:655] Throttling request took 1.048271953s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0416 00:46:02.725449 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:46:22.721932 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:46:34.375910 1 request.go:655] Throttling request took 1.048496301s, request: GET:https://192.168.76.2:8443/apis/batch/v1beta1?timeout=32s
W0416 00:46:35.227362 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:46:53.223961 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:47:06.877849 1 request.go:655] Throttling request took 1.04829265s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0416 00:47:07.729507 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:47:23.726273 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:47:39.380146 1 request.go:655] Throttling request took 1.048270054s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0416 00:47:40.231849 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:47:54.228390 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:48:11.882511 1 request.go:655] Throttling request took 1.048411725s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1?timeout=32s
W0416 00:48:12.733981 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0416 00:48:24.731323 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0416 00:48:44.384421 1 request.go:655] Throttling request took 1.048422269s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1beta1?timeout=32s
W0416 00:48:45.236309 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [bf3ceb2acadb] <==
I0416 00:40:41.627377 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0416 00:40:41.627660 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0416 00:40:41.893248 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0416 00:40:41.893347 1 server_others.go:185] Using iptables Proxier.
I0416 00:40:41.893556 1 server.go:650] Version: v1.20.0
I0416 00:40:41.894288 1 config.go:315] Starting service config controller
I0416 00:40:41.894309 1 shared_informer.go:240] Waiting for caches to sync for service config
I0416 00:40:41.898863 1 config.go:224] Starting endpoint slice config controller
I0416 00:40:41.898879 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0416 00:40:41.994410 1 shared_informer.go:247] Caches are synced for service config
I0416 00:40:41.999043 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [fa54eb276fa9] <==
I0416 00:43:03.639865 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0416 00:43:03.639973 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0416 00:43:03.686449 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0416 00:43:03.686583 1 server_others.go:185] Using iptables Proxier.
I0416 00:43:03.687094 1 server.go:650] Version: v1.20.0
I0416 00:43:03.687974 1 config.go:315] Starting service config controller
I0416 00:43:03.687994 1 shared_informer.go:240] Waiting for caches to sync for service config
I0416 00:43:03.688014 1 config.go:224] Starting endpoint slice config controller
I0416 00:43:03.688019 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0416 00:43:03.788175 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0416 00:43:03.788118 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [2d7d1b9e8353] <==
I0416 00:42:55.097742 1 serving.go:331] Generated self-signed cert in-memory
W0416 00:43:00.996853 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0416 00:43:00.996889 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0416 00:43:00.996902 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0416 00:43:00.996908 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0416 00:43:01.270542 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0416 00:43:01.270582 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0416 00:43:01.273092 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0416 00:43:01.273245 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0416 00:43:01.377695 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [7b437d823755] <==
W0416 00:40:17.341339 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0416 00:40:17.341389 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0416 00:40:17.341413 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0416 00:40:17.388261 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0416 00:40:17.388295 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0416 00:40:17.392734 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0416 00:40:17.392896 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0416 00:40:17.404568 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0416 00:40:17.409492 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0416 00:40:17.411893 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0416 00:40:17.414569 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0416 00:40:17.414995 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0416 00:40:17.418772 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0416 00:40:17.418796 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0416 00:40:17.423882 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0416 00:40:17.424021 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0416 00:40:17.424165 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0416 00:40:17.424258 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0416 00:40:17.444837 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0416 00:40:18.238624 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0416 00:40:18.254808 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0416 00:40:18.503867 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0416 00:40:18.510960 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0416 00:40:18.536747 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
I0416 00:40:18.888381 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Apr 16 00:46:30 old-k8s-version-014065 kubelet[1223]: E0416 00:46:30.931502 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:46:40 old-k8s-version-014065 kubelet[1223]: E0416 00:46:40.931581 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:46:45 old-k8s-version-014065 kubelet[1223]: E0416 00:46:45.971319 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:46:53 old-k8s-version-014065 kubelet[1223]: E0416 00:46:53.934295 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:46:56 old-k8s-version-014065 kubelet[1223]: E0416 00:46:56.930865 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:05 old-k8s-version-014065 kubelet[1223]: E0416 00:47:05.931075 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:10 old-k8s-version-014065 kubelet[1223]: E0416 00:47:10.931300 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:20 old-k8s-version-014065 kubelet[1223]: E0416 00:47:20.932206 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:25 old-k8s-version-014065 kubelet[1223]: E0416 00:47:25.931180 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:31 old-k8s-version-014065 kubelet[1223]: E0416 00:47:31.931916 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:37 old-k8s-version-014065 kubelet[1223]: E0416 00:47:37.931161 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:44 old-k8s-version-014065 kubelet[1223]: E0416 00:47:44.931376 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:48 old-k8s-version-014065 kubelet[1223]: E0416 00:47:48.937314 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:47:58 old-k8s-version-014065 kubelet[1223]: E0416 00:47:58.931861 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:03 old-k8s-version-014065 kubelet[1223]: E0416 00:48:03.931295 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:09 old-k8s-version-014065 kubelet[1223]: E0416 00:48:09.931027 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:18 old-k8s-version-014065 kubelet[1223]: E0416 00:48:18.930889 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:24 old-k8s-version-014065 kubelet[1223]: E0416 00:48:24.930944 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:33 old-k8s-version-014065 kubelet[1223]: E0416 00:48:33.935545 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:39 old-k8s-version-014065 kubelet[1223]: E0416 00:48:39.960832 1223 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Apr 16 00:48:39 old-k8s-version-014065 kubelet[1223]: E0416 00:48:39.960874 1223 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Apr 16 00:48:39 old-k8s-version-014065 kubelet[1223]: E0416 00:48:39.962361 1223 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-vbk9f,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exe
c:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-8k8tv_kube-system(660344
fa-e392-44df-a655-fe53ae49ca62): ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Apr 16 00:48:39 old-k8s-version-014065 kubelet[1223]: E0416 00:48:39.962405 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Apr 16 00:48:46 old-k8s-version-014065 kubelet[1223]: E0416 00:48:46.931162 1223 pod_workers.go:191] Error syncing pod 6b782303-4318-44b5-9f42-6664a7d0f2e5 ("dashboard-metrics-scraper-8d5bb5db8-6qzsp_kubernetes-dashboard(6b782303-4318-44b5-9f42-6664a7d0f2e5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
Apr 16 00:48:50 old-k8s-version-014065 kubelet[1223]: E0416 00:48:50.931517 1223 pod_workers.go:191] Error syncing pod 660344fa-e392-44df-a655-fe53ae49ca62 ("metrics-server-9975d5f86-8k8tv_kube-system(660344fa-e392-44df-a655-fe53ae49ca62)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
==> kubernetes-dashboard [c311fb93e11b] <==
2024/04/16 00:43:26 Using namespace: kubernetes-dashboard
2024/04/16 00:43:26 Using in-cluster config to connect to apiserver
2024/04/16 00:43:26 Using secret token for csrf signing
2024/04/16 00:43:26 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/04/16 00:43:26 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/04/16 00:43:26 Successful initial request to the apiserver, version: v1.20.0
2024/04/16 00:43:26 Generating JWE encryption key
2024/04/16 00:43:26 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/04/16 00:43:26 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/04/16 00:43:26 Initializing JWE encryption key from synchronized object
2024/04/16 00:43:26 Creating in-cluster Sidecar client
2024/04/16 00:43:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:43:26 Serving insecurely on HTTP port: 9090
2024/04/16 00:43:56 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:44:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:44:56 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:45:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:45:56 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:46:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:46:56 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:47:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:47:56 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:48:26 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/04/16 00:43:26 Starting overwatch
==> storage-provisioner [c5eb3f5fa95a] <==
I0416 00:43:49.094338 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0416 00:43:49.119390 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0416 00:43:49.119478 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0416 00:44:06.621990 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0416 00:44:06.622663 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"ff490061-67db-4c7b-9092-84836a794ef0", APIVersion:"v1", ResourceVersion:"803", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-014065_710b4ba9-0fee-4262-83b8-c20ff5069386 became leader
I0416 00:44:06.622910 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-014065_710b4ba9-0fee-4262-83b8-c20ff5069386!
I0416 00:44:06.723408 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-014065_710b4ba9-0fee-4262-83b8-c20ff5069386!
==> storage-provisioner [d8e8f85e95c4] <==
I0416 00:43:03.592305 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0416 00:43:33.594359 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-014065 -n old-k8s-version-014065
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-014065 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-8k8tv dashboard-metrics-scraper-8d5bb5db8-6qzsp
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-014065 describe pod metrics-server-9975d5f86-8k8tv dashboard-metrics-scraper-8d5bb5db8-6qzsp
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-014065 describe pod metrics-server-9975d5f86-8k8tv dashboard-metrics-scraper-8d5bb5db8-6qzsp: exit status 1 (96.059038ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-8k8tv" not found
Error from server (NotFound): pods "dashboard-metrics-scraper-8d5bb5db8-6qzsp" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-014065 describe pod metrics-server-9975d5f86-8k8tv dashboard-metrics-scraper-8d5bb5db8-6qzsp: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (371.23s)