=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-800769 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-800769 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m28.925352289s)
-- stdout --
* [old-k8s-version-800769] minikube v1.33.0-beta.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=18647
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18647-7542/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18647-7542/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-800769" primary control-plane node in "old-k8s-version-800769" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0416 00:50:14.742037 57518 out.go:291] Setting OutFile to fd 1 ...
I0416 00:50:14.742276 57518 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:50:14.742287 57518 out.go:304] Setting ErrFile to fd 2...
I0416 00:50:14.742291 57518 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0416 00:50:14.742500 57518 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18647-7542/.minikube/bin
I0416 00:50:14.743128 57518 out.go:298] Setting JSON to false
I0416 00:50:14.744169 57518 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":5559,"bootTime":1713223056,"procs":218,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0416 00:50:14.744227 57518 start.go:139] virtualization: kvm guest
I0416 00:50:14.746250 57518 out.go:177] * [old-k8s-version-800769] minikube v1.33.0-beta.0 on Ubuntu 20.04 (kvm/amd64)
I0416 00:50:14.747801 57518 out.go:177] - MINIKUBE_LOCATION=18647
I0416 00:50:14.747831 57518 notify.go:220] Checking for updates...
I0416 00:50:14.749303 57518 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0416 00:50:14.750681 57518 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18647-7542/kubeconfig
I0416 00:50:14.752098 57518 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18647-7542/.minikube
I0416 00:50:14.753531 57518 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0416 00:50:14.754639 57518 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0416 00:50:14.756330 57518 config.go:182] Loaded profile config "cert-expiration-359535": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0416 00:50:14.756421 57518 config.go:182] Loaded profile config "kubernetes-upgrade-497059": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.0-rc.2
I0416 00:50:14.756529 57518 config.go:182] Loaded profile config "pause-214771": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0416 00:50:14.756626 57518 driver.go:392] Setting default libvirt URI to qemu:///system
I0416 00:50:14.794372 57518 out.go:177] * Using the kvm2 driver based on user configuration
I0416 00:50:14.795566 57518 start.go:297] selected driver: kvm2
I0416 00:50:14.795580 57518 start.go:901] validating driver "kvm2" against <nil>
I0416 00:50:14.795592 57518 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0416 00:50:14.796324 57518 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0416 00:50:14.796398 57518 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/18647-7542/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0416 00:50:14.812045 57518 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.0-beta.0
I0416 00:50:14.812088 57518 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0416 00:50:14.812270 57518 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0416 00:50:14.812330 57518 cni.go:84] Creating CNI manager for ""
I0416 00:50:14.812342 57518 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0416 00:50:14.812351 57518 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0416 00:50:14.812399 57518 start.go:340] cluster config:
{Name:old-k8s-version-800769 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-800769 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:50:14.812489 57518 iso.go:125] acquiring lock: {Name:mk848ef90fbc2a1876645fc8fc16af382c3bcaa9 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0416 00:50:14.814187 57518 out.go:177] * Starting "old-k8s-version-800769" primary control-plane node in "old-k8s-version-800769" cluster
I0416 00:50:14.815477 57518 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0416 00:50:14.815509 57518 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0416 00:50:14.815519 57518 cache.go:56] Caching tarball of preloaded images
I0416 00:50:14.815610 57518 preload.go:173] Found /home/jenkins/minikube-integration/18647-7542/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0416 00:50:14.815621 57518 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0416 00:50:14.815702 57518 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/config.json ...
I0416 00:50:14.815718 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/config.json: {Name:mk188ed0158cdcdef6a943cf87c78b08b315b06c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:14.815830 57518 start.go:360] acquireMachinesLock for old-k8s-version-800769: {Name:mk92bff49461487f8cebf2747ccf61ccb9c772a2 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0416 00:50:14.815857 57518 start.go:364] duration metric: took 13.41µs to acquireMachinesLock for "old-k8s-version-800769"
I0416 00:50:14.815874 57518 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-800769 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18634/minikube-v1.33.0-1713175573-18634-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-800769 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0416 00:50:14.815956 57518 start.go:125] createHost starting for "" (driver="kvm2")
I0416 00:50:14.817627 57518 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0416 00:50:14.817754 57518 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0416 00:50:14.817785 57518 main.go:141] libmachine: Launching plugin server for driver kvm2
I0416 00:50:14.832633 57518 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:37955
I0416 00:50:14.833110 57518 main.go:141] libmachine: () Calling .GetVersion
I0416 00:50:14.833640 57518 main.go:141] libmachine: Using API Version 1
I0416 00:50:14.833678 57518 main.go:141] libmachine: () Calling .SetConfigRaw
I0416 00:50:14.834081 57518 main.go:141] libmachine: () Calling .GetMachineName
I0416 00:50:14.834342 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetMachineName
I0416 00:50:14.834529 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:14.834720 57518 start.go:159] libmachine.API.Create for "old-k8s-version-800769" (driver="kvm2")
I0416 00:50:14.834748 57518 client.go:168] LocalClient.Create starting
I0416 00:50:14.834785 57518 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem
I0416 00:50:14.834826 57518 main.go:141] libmachine: Decoding PEM data...
I0416 00:50:14.834847 57518 main.go:141] libmachine: Parsing certificate...
I0416 00:50:14.834929 57518 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18647-7542/.minikube/certs/cert.pem
I0416 00:50:14.834956 57518 main.go:141] libmachine: Decoding PEM data...
I0416 00:50:14.834982 57518 main.go:141] libmachine: Parsing certificate...
I0416 00:50:14.835007 57518 main.go:141] libmachine: Running pre-create checks...
I0416 00:50:14.835024 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .PreCreateCheck
I0416 00:50:14.835568 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetConfigRaw
I0416 00:50:14.836019 57518 main.go:141] libmachine: Creating machine...
I0416 00:50:14.836038 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .Create
I0416 00:50:14.836223 57518 main.go:141] libmachine: (old-k8s-version-800769) Creating KVM machine...
I0416 00:50:14.837473 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found existing default KVM network
I0416 00:50:14.838719 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.838562 57541 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr4 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:f6:f2:02} reservation:<nil>}
I0416 00:50:14.839456 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.839364 57541 network.go:211] skipping subnet 192.168.50.0/24 that is taken: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName:virbr2 IfaceIPv4:192.168.50.1 IfaceMTU:1500 IfaceMAC:52:54:00:8c:05:70} reservation:<nil>}
I0416 00:50:14.840234 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.840143 57541 network.go:211] skipping subnet 192.168.61.0/24 that is taken: &{IP:192.168.61.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.61.0/24 Gateway:192.168.61.1 ClientMin:192.168.61.2 ClientMax:192.168.61.254 Broadcast:192.168.61.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.61.1 IfaceMTU:1500 IfaceMAC:52:54:00:a2:7e:7e} reservation:<nil>}
I0416 00:50:14.842389 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.842255 57541 network.go:209] skipping subnet 192.168.72.0/24 that is reserved: &{IP:192.168.72.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.72.0/24 Gateway:192.168.72.1 ClientMin:192.168.72.2 ClientMax:192.168.72.254 Broadcast:192.168.72.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0416 00:50:14.843540 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.843452 57541 network.go:206] using free private subnet 192.168.83.0/24: &{IP:192.168.83.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.83.0/24 Gateway:192.168.83.1 ClientMin:192.168.83.2 ClientMax:192.168.83.254 Broadcast:192.168.83.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000112c30}
I0416 00:50:14.843594 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | created network xml:
I0416 00:50:14.843622 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <network>
I0416 00:50:14.843637 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <name>mk-old-k8s-version-800769</name>
I0416 00:50:14.843660 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <dns enable='no'/>
I0416 00:50:14.843687 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG |
I0416 00:50:14.843711 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <ip address='192.168.83.1' netmask='255.255.255.0'>
I0416 00:50:14.843724 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <dhcp>
I0416 00:50:14.843737 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | <range start='192.168.83.2' end='192.168.83.253'/>
I0416 00:50:14.843761 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | </dhcp>
I0416 00:50:14.843770 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | </ip>
I0416 00:50:14.843779 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG |
I0416 00:50:14.843794 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | </network>
I0416 00:50:14.843823 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG |
I0416 00:50:14.848837 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | trying to create private KVM network mk-old-k8s-version-800769 192.168.83.0/24...
I0416 00:50:14.915063 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | private KVM network mk-old-k8s-version-800769 192.168.83.0/24 created
I0416 00:50:14.915099 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:14.915037 57541 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/18647-7542/.minikube
I0416 00:50:14.915112 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting up store path in /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769 ...
I0416 00:50:14.915129 57518 main.go:141] libmachine: (old-k8s-version-800769) Building disk image from file:///home/jenkins/minikube-integration/18647-7542/.minikube/cache/iso/amd64/minikube-v1.33.0-1713175573-18634-amd64.iso
I0416 00:50:14.915188 57518 main.go:141] libmachine: (old-k8s-version-800769) Downloading /home/jenkins/minikube-integration/18647-7542/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/18647-7542/.minikube/cache/iso/amd64/minikube-v1.33.0-1713175573-18634-amd64.iso...
I0416 00:50:15.135863 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:15.135715 57541 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa...
I0416 00:50:15.308682 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:15.308573 57541 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/old-k8s-version-800769.rawdisk...
I0416 00:50:15.308709 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Writing magic tar header
I0416 00:50:15.308722 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Writing SSH key tar header
I0416 00:50:15.308730 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:15.308688 57541 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769 ...
I0416 00:50:15.308812 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769
I0416 00:50:15.308832 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18647-7542/.minikube/machines
I0416 00:50:15.308841 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769 (perms=drwx------)
I0416 00:50:15.308850 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18647-7542/.minikube
I0416 00:50:15.308860 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18647-7542
I0416 00:50:15.308868 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0416 00:50:15.308877 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins/minikube-integration/18647-7542/.minikube/machines (perms=drwxr-xr-x)
I0416 00:50:15.308888 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins/minikube-integration/18647-7542/.minikube (perms=drwxr-xr-x)
I0416 00:50:15.308897 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins/minikube-integration/18647-7542 (perms=drwxrwxr-x)
I0416 00:50:15.308909 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0416 00:50:15.308916 57518 main.go:141] libmachine: (old-k8s-version-800769) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0416 00:50:15.308922 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home/jenkins
I0416 00:50:15.308930 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Checking permissions on dir: /home
I0416 00:50:15.308936 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Skipping /home - not owner
I0416 00:50:15.308945 57518 main.go:141] libmachine: (old-k8s-version-800769) Creating domain...
I0416 00:50:15.310097 57518 main.go:141] libmachine: (old-k8s-version-800769) define libvirt domain using xml:
I0416 00:50:15.310116 57518 main.go:141] libmachine: (old-k8s-version-800769) <domain type='kvm'>
I0416 00:50:15.310124 57518 main.go:141] libmachine: (old-k8s-version-800769) <name>old-k8s-version-800769</name>
I0416 00:50:15.310129 57518 main.go:141] libmachine: (old-k8s-version-800769) <memory unit='MiB'>2200</memory>
I0416 00:50:15.310135 57518 main.go:141] libmachine: (old-k8s-version-800769) <vcpu>2</vcpu>
I0416 00:50:15.310155 57518 main.go:141] libmachine: (old-k8s-version-800769) <features>
I0416 00:50:15.310169 57518 main.go:141] libmachine: (old-k8s-version-800769) <acpi/>
I0416 00:50:15.310176 57518 main.go:141] libmachine: (old-k8s-version-800769) <apic/>
I0416 00:50:15.310183 57518 main.go:141] libmachine: (old-k8s-version-800769) <pae/>
I0416 00:50:15.310188 57518 main.go:141] libmachine: (old-k8s-version-800769)
I0416 00:50:15.310194 57518 main.go:141] libmachine: (old-k8s-version-800769) </features>
I0416 00:50:15.310202 57518 main.go:141] libmachine: (old-k8s-version-800769) <cpu mode='host-passthrough'>
I0416 00:50:15.310207 57518 main.go:141] libmachine: (old-k8s-version-800769)
I0416 00:50:15.310215 57518 main.go:141] libmachine: (old-k8s-version-800769) </cpu>
I0416 00:50:15.310236 57518 main.go:141] libmachine: (old-k8s-version-800769) <os>
I0416 00:50:15.310256 57518 main.go:141] libmachine: (old-k8s-version-800769) <type>hvm</type>
I0416 00:50:15.310263 57518 main.go:141] libmachine: (old-k8s-version-800769) <boot dev='cdrom'/>
I0416 00:50:15.310269 57518 main.go:141] libmachine: (old-k8s-version-800769) <boot dev='hd'/>
I0416 00:50:15.310278 57518 main.go:141] libmachine: (old-k8s-version-800769) <bootmenu enable='no'/>
I0416 00:50:15.310283 57518 main.go:141] libmachine: (old-k8s-version-800769) </os>
I0416 00:50:15.310291 57518 main.go:141] libmachine: (old-k8s-version-800769) <devices>
I0416 00:50:15.310297 57518 main.go:141] libmachine: (old-k8s-version-800769) <disk type='file' device='cdrom'>
I0416 00:50:15.310308 57518 main.go:141] libmachine: (old-k8s-version-800769) <source file='/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/boot2docker.iso'/>
I0416 00:50:15.310313 57518 main.go:141] libmachine: (old-k8s-version-800769) <target dev='hdc' bus='scsi'/>
I0416 00:50:15.310319 57518 main.go:141] libmachine: (old-k8s-version-800769) <readonly/>
I0416 00:50:15.310327 57518 main.go:141] libmachine: (old-k8s-version-800769) </disk>
I0416 00:50:15.310337 57518 main.go:141] libmachine: (old-k8s-version-800769) <disk type='file' device='disk'>
I0416 00:50:15.310351 57518 main.go:141] libmachine: (old-k8s-version-800769) <driver name='qemu' type='raw' cache='default' io='threads' />
I0416 00:50:15.310368 57518 main.go:141] libmachine: (old-k8s-version-800769) <source file='/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/old-k8s-version-800769.rawdisk'/>
I0416 00:50:15.310380 57518 main.go:141] libmachine: (old-k8s-version-800769) <target dev='hda' bus='virtio'/>
I0416 00:50:15.310389 57518 main.go:141] libmachine: (old-k8s-version-800769) </disk>
I0416 00:50:15.310401 57518 main.go:141] libmachine: (old-k8s-version-800769) <interface type='network'>
I0416 00:50:15.310415 57518 main.go:141] libmachine: (old-k8s-version-800769) <source network='mk-old-k8s-version-800769'/>
I0416 00:50:15.310424 57518 main.go:141] libmachine: (old-k8s-version-800769) <model type='virtio'/>
I0416 00:50:15.310456 57518 main.go:141] libmachine: (old-k8s-version-800769) </interface>
I0416 00:50:15.310476 57518 main.go:141] libmachine: (old-k8s-version-800769) <interface type='network'>
I0416 00:50:15.310488 57518 main.go:141] libmachine: (old-k8s-version-800769) <source network='default'/>
I0416 00:50:15.310500 57518 main.go:141] libmachine: (old-k8s-version-800769) <model type='virtio'/>
I0416 00:50:15.310511 57518 main.go:141] libmachine: (old-k8s-version-800769) </interface>
I0416 00:50:15.310527 57518 main.go:141] libmachine: (old-k8s-version-800769) <serial type='pty'>
I0416 00:50:15.310541 57518 main.go:141] libmachine: (old-k8s-version-800769) <target port='0'/>
I0416 00:50:15.310553 57518 main.go:141] libmachine: (old-k8s-version-800769) </serial>
I0416 00:50:15.310567 57518 main.go:141] libmachine: (old-k8s-version-800769) <console type='pty'>
I0416 00:50:15.310579 57518 main.go:141] libmachine: (old-k8s-version-800769) <target type='serial' port='0'/>
I0416 00:50:15.310592 57518 main.go:141] libmachine: (old-k8s-version-800769) </console>
I0416 00:50:15.310608 57518 main.go:141] libmachine: (old-k8s-version-800769) <rng model='virtio'>
I0416 00:50:15.310624 57518 main.go:141] libmachine: (old-k8s-version-800769) <backend model='random'>/dev/random</backend>
I0416 00:50:15.310636 57518 main.go:141] libmachine: (old-k8s-version-800769) </rng>
I0416 00:50:15.310649 57518 main.go:141] libmachine: (old-k8s-version-800769)
I0416 00:50:15.310660 57518 main.go:141] libmachine: (old-k8s-version-800769)
I0416 00:50:15.310670 57518 main.go:141] libmachine: (old-k8s-version-800769) </devices>
I0416 00:50:15.310685 57518 main.go:141] libmachine: (old-k8s-version-800769) </domain>
I0416 00:50:15.310699 57518 main.go:141] libmachine: (old-k8s-version-800769)
I0416 00:50:15.314754 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a0:db:07 in network default
I0416 00:50:15.315363 57518 main.go:141] libmachine: (old-k8s-version-800769) Ensuring networks are active...
I0416 00:50:15.315390 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:15.316041 57518 main.go:141] libmachine: (old-k8s-version-800769) Ensuring network default is active
I0416 00:50:15.316433 57518 main.go:141] libmachine: (old-k8s-version-800769) Ensuring network mk-old-k8s-version-800769 is active
I0416 00:50:15.317088 57518 main.go:141] libmachine: (old-k8s-version-800769) Getting domain xml...
I0416 00:50:15.317857 57518 main.go:141] libmachine: (old-k8s-version-800769) Creating domain...
I0416 00:50:16.617402 57518 main.go:141] libmachine: (old-k8s-version-800769) Waiting to get IP...
I0416 00:50:16.618172 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:16.618730 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:16.618788 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:16.618717 57541 retry.go:31] will retry after 245.38194ms: waiting for machine to come up
I0416 00:50:16.866167 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:16.866699 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:16.866726 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:16.866659 57541 retry.go:31] will retry after 276.679462ms: waiting for machine to come up
I0416 00:50:17.145220 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:17.145785 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:17.145817 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:17.145732 57541 retry.go:31] will retry after 462.312932ms: waiting for machine to come up
I0416 00:50:17.609346 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:17.609922 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:17.609951 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:17.609880 57541 retry.go:31] will retry after 444.438479ms: waiting for machine to come up
I0416 00:50:18.055580 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:18.056026 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:18.056054 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:18.056000 57541 retry.go:31] will retry after 651.351774ms: waiting for machine to come up
I0416 00:50:18.708903 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:18.709388 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:18.709421 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:18.709351 57541 retry.go:31] will retry after 639.944351ms: waiting for machine to come up
I0416 00:50:19.350987 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:19.351503 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:19.351529 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:19.351457 57541 retry.go:31] will retry after 715.915795ms: waiting for machine to come up
I0416 00:50:20.069287 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:20.069873 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:20.069901 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:20.069854 57541 retry.go:31] will retry after 1.140237119s: waiting for machine to come up
I0416 00:50:21.211964 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:21.212558 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:21.212586 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:21.212505 57541 retry.go:31] will retry after 1.276242671s: waiting for machine to come up
I0416 00:50:22.490513 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:22.491047 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:22.491076 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:22.491003 57541 retry.go:31] will retry after 1.525621794s: waiting for machine to come up
I0416 00:50:24.018790 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:24.019320 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:24.019352 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:24.019275 57541 retry.go:31] will retry after 2.192801459s: waiting for machine to come up
I0416 00:50:26.214035 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:26.214705 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:26.214737 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:26.214663 57541 retry.go:31] will retry after 3.606824053s: waiting for machine to come up
I0416 00:50:29.825017 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:29.825506 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:29.825539 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:29.825461 57541 retry.go:31] will retry after 3.080343871s: waiting for machine to come up
I0416 00:50:32.909816 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:32.910470 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find current IP address of domain old-k8s-version-800769 in network mk-old-k8s-version-800769
I0416 00:50:32.910494 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | I0416 00:50:32.910430 57541 retry.go:31] will retry after 3.609189483s: waiting for machine to come up
I0416 00:50:36.521773 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.522355 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has current primary IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.522392 57518 main.go:141] libmachine: (old-k8s-version-800769) Found IP for machine: 192.168.83.98
I0416 00:50:36.522406 57518 main.go:141] libmachine: (old-k8s-version-800769) Reserving static IP address...
I0416 00:50:36.522808 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-800769", mac: "52:54:00:a1:ad:da", ip: "192.168.83.98"} in network mk-old-k8s-version-800769
I0416 00:50:36.600263 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Getting to WaitForSSH function...
I0416 00:50:36.600296 57518 main.go:141] libmachine: (old-k8s-version-800769) Reserved static IP address: 192.168.83.98
I0416 00:50:36.600311 57518 main.go:141] libmachine: (old-k8s-version-800769) Waiting for SSH to be available...
I0416 00:50:36.603616 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.604147 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:minikube Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:36.604174 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.604366 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Using SSH client type: external
I0416 00:50:36.604392 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Using SSH private key: /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa (-rw-------)
I0416 00:50:36.604420 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.83.98 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa -p 22] /usr/bin/ssh <nil>}
I0416 00:50:36.604447 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | About to run SSH command:
I0416 00:50:36.604475 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | exit 0
I0416 00:50:36.733728 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | SSH cmd err, output: <nil>:
I0416 00:50:36.733979 57518 main.go:141] libmachine: (old-k8s-version-800769) KVM machine creation complete!
I0416 00:50:36.734318 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetConfigRaw
I0416 00:50:36.734927 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:36.735159 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:36.735323 57518 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0416 00:50:36.735337 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetState
I0416 00:50:36.736816 57518 main.go:141] libmachine: Detecting operating system of created instance...
I0416 00:50:36.736833 57518 main.go:141] libmachine: Waiting for SSH to be available...
I0416 00:50:36.736841 57518 main.go:141] libmachine: Getting to WaitForSSH function...
I0416 00:50:36.736848 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:36.739586 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.739924 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:36.739960 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.740072 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:36.740252 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.740422 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.740589 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:36.740761 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:36.740976 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:36.740988 57518 main.go:141] libmachine: About to run SSH command:
exit 0
I0416 00:50:36.857473 57518 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:50:36.857510 57518 main.go:141] libmachine: Detecting the provisioner...
I0416 00:50:36.857521 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:36.863977 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.864338 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:36.864434 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.864508 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:36.864706 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.864877 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.865022 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:36.865210 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:36.865392 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:36.865404 57518 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0416 00:50:36.986081 57518 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0416 00:50:36.986137 57518 main.go:141] libmachine: found compatible host: buildroot
I0416 00:50:36.986144 57518 main.go:141] libmachine: Provisioning with buildroot...
I0416 00:50:36.986151 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetMachineName
I0416 00:50:36.986386 57518 buildroot.go:166] provisioning hostname "old-k8s-version-800769"
I0416 00:50:36.986417 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetMachineName
I0416 00:50:36.986613 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:36.989366 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.989859 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:36.989894 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:36.989984 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:36.990191 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.990383 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:36.990588 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:36.990803 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:36.991032 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:36.991050 57518 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-800769 && echo "old-k8s-version-800769" | sudo tee /etc/hostname
I0416 00:50:37.124891 57518 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-800769
I0416 00:50:37.124944 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:37.533517 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.533935 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:37.533965 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.534246 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:37.534491 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:37.534695 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:37.534872 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:37.535049 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:37.535222 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:37.535238 57518 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-800769' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-800769/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-800769' | sudo tee -a /etc/hosts;
fi
fi
I0416 00:50:37.668083 57518 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0416 00:50:37.668119 57518 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/18647-7542/.minikube CaCertPath:/home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18647-7542/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18647-7542/.minikube}
I0416 00:50:37.668144 57518 buildroot.go:174] setting up certificates
I0416 00:50:37.668158 57518 provision.go:84] configureAuth start
I0416 00:50:37.668169 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetMachineName
I0416 00:50:37.668416 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetIP
I0416 00:50:37.671146 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.671488 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:37.671513 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.671695 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:37.674056 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.674360 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:37.674390 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.674552 57518 provision.go:143] copyHostCerts
I0416 00:50:37.674618 57518 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-7542/.minikube/key.pem, removing ...
I0416 00:50:37.674639 57518 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-7542/.minikube/key.pem
I0416 00:50:37.674714 57518 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18647-7542/.minikube/key.pem (1675 bytes)
I0416 00:50:37.674818 57518 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-7542/.minikube/ca.pem, removing ...
I0416 00:50:37.674827 57518 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-7542/.minikube/ca.pem
I0416 00:50:37.674855 57518 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18647-7542/.minikube/ca.pem (1082 bytes)
I0416 00:50:37.674916 57518 exec_runner.go:144] found /home/jenkins/minikube-integration/18647-7542/.minikube/cert.pem, removing ...
I0416 00:50:37.674923 57518 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18647-7542/.minikube/cert.pem
I0416 00:50:37.674944 57518 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18647-7542/.minikube/cert.pem (1123 bytes)
I0416 00:50:37.675006 57518 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18647-7542/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-800769 san=[127.0.0.1 192.168.83.98 localhost minikube old-k8s-version-800769]
I0416 00:50:37.761500 57518 provision.go:177] copyRemoteCerts
I0416 00:50:37.761549 57518 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0416 00:50:37.761570 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:37.764418 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.764815 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:37.764863 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.765244 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:37.765441 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:37.765652 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:37.765820 57518 sshutil.go:53] new ssh client: &{IP:192.168.83.98 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa Username:docker}
I0416 00:50:37.859684 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0416 00:50:37.885008 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0416 00:50:37.912639 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0416 00:50:37.939479 57518 provision.go:87] duration metric: took 271.310199ms to configureAuth
I0416 00:50:37.939505 57518 buildroot.go:189] setting minikube options for container-runtime
I0416 00:50:37.939694 57518 config.go:182] Loaded profile config "old-k8s-version-800769": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0416 00:50:37.939776 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:37.942573 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.942962 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:37.942998 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:37.943180 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:37.943378 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:37.943565 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:37.943694 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:37.943860 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:37.944016 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:37.944032 57518 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0416 00:50:38.248545 57518 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0416 00:50:38.248570 57518 main.go:141] libmachine: Checking connection to Docker...
I0416 00:50:38.248580 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetURL
I0416 00:50:38.249965 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | Using libvirt version 6000000
I0416 00:50:38.252812 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.253188 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.253216 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.253397 57518 main.go:141] libmachine: Docker is up and running!
I0416 00:50:38.253412 57518 main.go:141] libmachine: Reticulating splines...
I0416 00:50:38.253420 57518 client.go:171] duration metric: took 23.418664074s to LocalClient.Create
I0416 00:50:38.253441 57518 start.go:167] duration metric: took 23.418722474s to libmachine.API.Create "old-k8s-version-800769"
I0416 00:50:38.253453 57518 start.go:293] postStartSetup for "old-k8s-version-800769" (driver="kvm2")
I0416 00:50:38.253465 57518 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0416 00:50:38.253485 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:38.253688 57518 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0416 00:50:38.253710 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:38.255706 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.256006 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.256025 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.256160 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:38.256336 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:38.256496 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:38.256644 57518 sshutil.go:53] new ssh client: &{IP:192.168.83.98 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa Username:docker}
I0416 00:50:38.349581 57518 ssh_runner.go:195] Run: cat /etc/os-release
I0416 00:50:38.354233 57518 info.go:137] Remote host: Buildroot 2023.02.9
I0416 00:50:38.354253 57518 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-7542/.minikube/addons for local assets ...
I0416 00:50:38.354324 57518 filesync.go:126] Scanning /home/jenkins/minikube-integration/18647-7542/.minikube/files for local assets ...
I0416 00:50:38.354442 57518 filesync.go:149] local asset: /home/jenkins/minikube-integration/18647-7542/.minikube/files/etc/ssl/certs/148972.pem -> 148972.pem in /etc/ssl/certs
I0416 00:50:38.354558 57518 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0416 00:50:38.365958 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/files/etc/ssl/certs/148972.pem --> /etc/ssl/certs/148972.pem (1708 bytes)
I0416 00:50:38.391649 57518 start.go:296] duration metric: took 138.183727ms for postStartSetup
I0416 00:50:38.391700 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetConfigRaw
I0416 00:50:38.392352 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetIP
I0416 00:50:38.395003 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.395352 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.395390 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.395651 57518 profile.go:143] Saving config to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/config.json ...
I0416 00:50:38.395871 57518 start.go:128] duration metric: took 23.579899593s to createHost
I0416 00:50:38.395899 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:38.398267 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.398729 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.398750 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.398915 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:38.399112 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:38.399307 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:38.399449 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:38.399625 57518 main.go:141] libmachine: Using SSH client type: native
I0416 00:50:38.399812 57518 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4e0] 0x830240 <nil> [] 0s} 192.168.83.98 22 <nil> <nil>}
I0416 00:50:38.399824 57518 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0416 00:50:38.514068 57518 main.go:141] libmachine: SSH cmd err, output: <nil>: 1713228638.475939923
I0416 00:50:38.514091 57518 fix.go:216] guest clock: 1713228638.475939923
I0416 00:50:38.514100 57518 fix.go:229] Guest: 2024-04-16 00:50:38.475939923 +0000 UTC Remote: 2024-04-16 00:50:38.395885368 +0000 UTC m=+23.704181515 (delta=80.054555ms)
I0416 00:50:38.514122 57518 fix.go:200] guest clock delta is within tolerance: 80.054555ms
I0416 00:50:38.514128 57518 start.go:83] releasing machines lock for "old-k8s-version-800769", held for 23.698264667s
I0416 00:50:38.514158 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:38.514465 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetIP
I0416 00:50:38.517151 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.517538 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.517561 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.517759 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:38.518184 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:38.518366 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .DriverName
I0416 00:50:38.518460 57518 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0416 00:50:38.518496 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:38.518578 57518 ssh_runner.go:195] Run: cat /version.json
I0416 00:50:38.518595 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHHostname
I0416 00:50:38.521398 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.521629 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.521740 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.521763 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.521938 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:38.522039 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:38.522064 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:38.522092 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:38.522211 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHPort
I0416 00:50:38.522262 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:38.522382 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHKeyPath
I0416 00:50:38.522426 57518 sshutil.go:53] new ssh client: &{IP:192.168.83.98 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa Username:docker}
I0416 00:50:38.522521 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetSSHUsername
I0416 00:50:38.522637 57518 sshutil.go:53] new ssh client: &{IP:192.168.83.98 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18647-7542/.minikube/machines/old-k8s-version-800769/id_rsa Username:docker}
I0416 00:50:38.644730 57518 ssh_runner.go:195] Run: systemctl --version
I0416 00:50:38.653791 57518 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0416 00:50:38.813040 57518 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0416 00:50:38.820539 57518 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0416 00:50:38.820596 57518 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0416 00:50:38.837858 57518 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0416 00:50:38.837878 57518 start.go:494] detecting cgroup driver to use...
I0416 00:50:38.837942 57518 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0416 00:50:38.857063 57518 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0416 00:50:38.871590 57518 docker.go:217] disabling cri-docker service (if available) ...
I0416 00:50:38.871651 57518 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0416 00:50:38.885523 57518 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0416 00:50:38.899251 57518 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0416 00:50:39.038525 57518 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0416 00:50:39.188520 57518 docker.go:233] disabling docker service ...
I0416 00:50:39.188599 57518 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0416 00:50:39.204403 57518 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0416 00:50:39.218651 57518 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0416 00:50:39.360941 57518 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0416 00:50:39.484295 57518 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0416 00:50:39.499266 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0416 00:50:39.518776 57518 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0416 00:50:39.518855 57518 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0416 00:50:39.530420 57518 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0416 00:50:39.530490 57518 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0416 00:50:39.542307 57518 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0416 00:50:39.556149 57518 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0416 00:50:39.567354 57518 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0416 00:50:39.580456 57518 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0416 00:50:39.591930 57518 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0416 00:50:39.591978 57518 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0416 00:50:39.608453 57518 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0416 00:50:39.619913 57518 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:50:39.762707 57518 ssh_runner.go:195] Run: sudo systemctl restart crio
I0416 00:50:39.932449 57518 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0416 00:50:39.932537 57518 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0416 00:50:39.937614 57518 start.go:562] Will wait 60s for crictl version
I0416 00:50:39.937672 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:39.941721 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0416 00:50:39.979406 57518 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0416 00:50:39.979505 57518 ssh_runner.go:195] Run: crio --version
I0416 00:50:40.016207 57518 ssh_runner.go:195] Run: crio --version
I0416 00:50:40.049242 57518 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0416 00:50:40.050503 57518 main.go:141] libmachine: (old-k8s-version-800769) Calling .GetIP
I0416 00:50:40.053317 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:40.053665 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a1:ad:da", ip: ""} in network mk-old-k8s-version-800769: {Iface:virbr3 ExpiryTime:2024-04-16 01:50:30 +0000 UTC Type:0 Mac:52:54:00:a1:ad:da Iaid: IPaddr:192.168.83.98 Prefix:24 Hostname:old-k8s-version-800769 Clientid:01:52:54:00:a1:ad:da}
I0416 00:50:40.053693 57518 main.go:141] libmachine: (old-k8s-version-800769) DBG | domain old-k8s-version-800769 has defined IP address 192.168.83.98 and MAC address 52:54:00:a1:ad:da in network mk-old-k8s-version-800769
I0416 00:50:40.053885 57518 ssh_runner.go:195] Run: grep 192.168.83.1 host.minikube.internal$ /etc/hosts
I0416 00:50:40.059572 57518 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.83.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:50:40.077817 57518 kubeadm.go:877] updating cluster {Name:old-k8s-version-800769 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18634/minikube-v1.33.0-1713175573-18634-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-800769 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.83.98 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0416 00:50:40.077946 57518 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0416 00:50:40.077993 57518 ssh_runner.go:195] Run: sudo crictl images --output json
I0416 00:50:40.112737 57518 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0416 00:50:40.112852 57518 ssh_runner.go:195] Run: which lz4
I0416 00:50:40.119292 57518 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0416 00:50:40.126098 57518 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0416 00:50:40.126134 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0416 00:50:41.951076 57518 crio.go:462] duration metric: took 1.831812475s to copy over tarball
I0416 00:50:41.951156 57518 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0416 00:50:44.628845 57518 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.677607548s)
I0416 00:50:44.628891 57518 crio.go:469] duration metric: took 2.67778437s to extract the tarball
I0416 00:50:44.628902 57518 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0416 00:50:44.670960 57518 ssh_runner.go:195] Run: sudo crictl images --output json
I0416 00:50:44.719857 57518 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0416 00:50:44.719880 57518 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0416 00:50:44.719934 57518 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0416 00:50:44.719961 57518 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0416 00:50:44.719971 57518 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0416 00:50:44.719990 57518 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0416 00:50:44.720016 57518 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0416 00:50:44.719974 57518 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0416 00:50:44.720109 57518 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0416 00:50:44.720231 57518 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0416 00:50:44.721516 57518 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0416 00:50:44.721611 57518 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0416 00:50:44.721723 57518 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0416 00:50:44.721825 57518 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0416 00:50:44.721948 57518 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0416 00:50:44.722088 57518 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0416 00:50:44.722624 57518 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0416 00:50:44.722793 57518 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0416 00:50:44.915941 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0416 00:50:44.920591 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0416 00:50:44.925481 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0416 00:50:44.933916 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0416 00:50:44.956127 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0416 00:50:44.956648 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0416 00:50:44.988848 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0416 00:50:44.998379 57518 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0416 00:50:44.998422 57518 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0416 00:50:44.998468 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.077356 57518 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0416 00:50:45.077399 57518 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0416 00:50:45.077438 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.104861 57518 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0416 00:50:45.104907 57518 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0416 00:50:45.104958 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.108866 57518 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0416 00:50:45.108910 57518 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0416 00:50:45.108961 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.126401 57518 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0416 00:50:45.126439 57518 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0416 00:50:45.126452 57518 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0416 00:50:45.126470 57518 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0416 00:50:45.126502 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.126508 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.138458 57518 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0416 00:50:45.138499 57518 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0416 00:50:45.138530 57518 ssh_runner.go:195] Run: which crictl
I0416 00:50:45.138538 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0416 00:50:45.138612 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0416 00:50:45.138653 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0416 00:50:45.138739 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0416 00:50:45.138820 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0416 00:50:45.138856 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0416 00:50:45.229445 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0416 00:50:45.229834 57518 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0416 00:50:45.294896 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0416 00:50:45.294919 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0416 00:50:45.294987 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0416 00:50:45.295024 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0416 00:50:45.295101 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0416 00:50:45.308884 57518 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0416 00:50:45.539557 57518 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0416 00:50:45.689006 57518 cache_images.go:92] duration metric: took 969.109018ms to LoadCachedImages
W0416 00:50:45.689094 57518 out.go:239] X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0: no such file or directory
X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18647-7542/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0: no such file or directory
I0416 00:50:45.689111 57518 kubeadm.go:928] updating node { 192.168.83.98 8443 v1.20.0 crio true true} ...
I0416 00:50:45.689260 57518 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-800769 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.83.98
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-800769 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0416 00:50:45.689353 57518 ssh_runner.go:195] Run: crio config
I0416 00:50:45.742458 57518 cni.go:84] Creating CNI manager for ""
I0416 00:50:45.742481 57518 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0416 00:50:45.742489 57518 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0416 00:50:45.742506 57518 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.83.98 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-800769 NodeName:old-k8s-version-800769 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.83.98"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.83.98 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0416 00:50:45.742647 57518 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.83.98
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-800769"
kubeletExtraArgs:
node-ip: 192.168.83.98
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.83.98"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0416 00:50:45.742717 57518 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0416 00:50:45.754416 57518 binaries.go:44] Found k8s binaries, skipping transfer
I0416 00:50:45.754539 57518 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0416 00:50:45.766037 57518 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0416 00:50:45.785952 57518 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0416 00:50:45.805087 57518 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0416 00:50:45.824517 57518 ssh_runner.go:195] Run: grep 192.168.83.98 control-plane.minikube.internal$ /etc/hosts
I0416 00:50:45.828864 57518 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.83.98 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0416 00:50:45.843314 57518 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0416 00:50:45.966292 57518 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0416 00:50:45.989009 57518 certs.go:68] Setting up /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769 for IP: 192.168.83.98
I0416 00:50:45.989033 57518 certs.go:194] generating shared ca certs ...
I0416 00:50:45.989053 57518 certs.go:226] acquiring lock for ca certs: {Name:mkcfa1570e683d94647c63485e1bbb8cf0788316 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:45.989227 57518 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18647-7542/.minikube/ca.key
I0416 00:50:45.989293 57518 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18647-7542/.minikube/proxy-client-ca.key
I0416 00:50:45.989315 57518 certs.go:256] generating profile certs ...
I0416 00:50:45.989406 57518 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.key
I0416 00:50:45.989425 57518 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.crt with IP's: []
I0416 00:50:46.408585 57518 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.crt ...
I0416 00:50:46.408617 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.crt: {Name:mkbf676d8c3ddc7b08887a91226d9a5b8803a15b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.408774 57518 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.key ...
I0416 00:50:46.408788 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/client.key: {Name:mk9591e9503c1363e321a05f9763382597a7b2a7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.408858 57518 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key.efc35655
I0416 00:50:46.408875 57518 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt.efc35655 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.83.98]
I0416 00:50:46.476105 57518 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt.efc35655 ...
I0416 00:50:46.476137 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt.efc35655: {Name:mk51f94e119f1853fcd97c9c998d1ac92cf56c3d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.476296 57518 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key.efc35655 ...
I0416 00:50:46.476313 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key.efc35655: {Name:mk16afa9da68a71ff2bc6215134e749647d05aa6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.476390 57518 certs.go:381] copying /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt.efc35655 -> /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt
I0416 00:50:46.476504 57518 certs.go:385] copying /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key.efc35655 -> /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key
I0416 00:50:46.476561 57518 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.key
I0416 00:50:46.476577 57518 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.crt with IP's: []
I0416 00:50:46.579628 57518 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.crt ...
I0416 00:50:46.579659 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.crt: {Name:mk36ff5ae542f3f52e9086043c7eb20368769717 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.579839 57518 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.key ...
I0416 00:50:46.579858 57518 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.key: {Name:mk35fd4ce01a10979b5964145582219381db20b4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0416 00:50:46.580015 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/14897.pem (1338 bytes)
W0416 00:50:46.580050 57518 certs.go:480] ignoring /home/jenkins/minikube-integration/18647-7542/.minikube/certs/14897_empty.pem, impossibly tiny 0 bytes
I0416 00:50:46.580058 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca-key.pem (1679 bytes)
I0416 00:50:46.580081 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/ca.pem (1082 bytes)
I0416 00:50:46.580103 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/cert.pem (1123 bytes)
I0416 00:50:46.580127 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/certs/key.pem (1675 bytes)
I0416 00:50:46.580166 57518 certs.go:484] found cert: /home/jenkins/minikube-integration/18647-7542/.minikube/files/etc/ssl/certs/148972.pem (1708 bytes)
I0416 00:50:46.580851 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0416 00:50:46.609425 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0416 00:50:46.635400 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0416 00:50:46.661824 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0416 00:50:46.689615 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0416 00:50:46.716663 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0416 00:50:46.744772 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0416 00:50:46.770804 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/profiles/old-k8s-version-800769/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0416 00:50:46.802804 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/certs/14897.pem --> /usr/share/ca-certificates/14897.pem (1338 bytes)
I0416 00:50:46.839736 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/files/etc/ssl/certs/148972.pem --> /usr/share/ca-certificates/148972.pem (1708 bytes)
I0416 00:50:46.874757 57518 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18647-7542/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0416 00:50:46.910367 57518 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0416 00:50:46.928703 57518 ssh_runner.go:195] Run: openssl version
I0416 00:50:46.935429 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/14897.pem && ln -fs /usr/share/ca-certificates/14897.pem /etc/ssl/certs/14897.pem"
I0416 00:50:46.948464 57518 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/14897.pem
I0416 00:50:46.954456 57518 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Apr 15 23:49 /usr/share/ca-certificates/14897.pem
I0416 00:50:46.954522 57518 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/14897.pem
I0416 00:50:46.960795 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/14897.pem /etc/ssl/certs/51391683.0"
I0416 00:50:46.972975 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/148972.pem && ln -fs /usr/share/ca-certificates/148972.pem /etc/ssl/certs/148972.pem"
I0416 00:50:46.985038 57518 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/148972.pem
I0416 00:50:46.990576 57518 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Apr 15 23:49 /usr/share/ca-certificates/148972.pem
I0416 00:50:46.990645 57518 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/148972.pem
I0416 00:50:46.997729 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/148972.pem /etc/ssl/certs/3ec20f2e.0"
I0416 00:50:47.010405 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0416 00:50:47.022045 57518 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0416 00:50:47.027031 57518 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Apr 15 23:39 /usr/share/ca-certificates/minikubeCA.pem
I0416 00:50:47.027089 57518 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0416 00:50:47.033173 57518 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0416 00:50:47.044402 57518 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0416 00:50:47.048979 57518 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0416 00:50:47.049041 57518 kubeadm.go:391] StartCluster: {Name:old-k8s-version-800769 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18634/minikube-v1.33.0-1713175573-18634-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1713215244-18647@sha256:4eb69c9ed3e92807cea9443b515ec5d46db84479de7669694de8c98e2d40c4af Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-800769 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.83.98 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0416 00:50:47.049105 57518 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0416 00:50:47.049145 57518 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0416 00:50:47.090049 57518 cri.go:89] found id: ""
I0416 00:50:47.090109 57518 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0416 00:50:47.100864 57518 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0416 00:50:47.111047 57518 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0416 00:50:47.121411 57518 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0416 00:50:47.121437 57518 kubeadm.go:156] found existing configuration files:
I0416 00:50:47.121508 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0416 00:50:47.131033 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0416 00:50:47.131097 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0416 00:50:47.141545 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0416 00:50:47.151231 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0416 00:50:47.151293 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0416 00:50:47.161764 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0416 00:50:47.172124 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0416 00:50:47.172190 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0416 00:50:47.182283 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0416 00:50:47.192217 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0416 00:50:47.192282 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0416 00:50:47.202015 57518 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0416 00:50:47.317969 57518 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0416 00:50:47.318060 57518 kubeadm.go:309] [preflight] Running pre-flight checks
I0416 00:50:47.475619 57518 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0416 00:50:47.475746 57518 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0416 00:50:47.475894 57518 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0416 00:50:47.670930 57518 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0416 00:50:47.673739 57518 out.go:204] - Generating certificates and keys ...
I0416 00:50:47.673880 57518 kubeadm.go:309] [certs] Using existing ca certificate authority
I0416 00:50:47.673996 57518 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0416 00:50:47.747292 57518 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0416 00:50:47.931730 57518 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0416 00:50:48.040395 57518 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0416 00:50:48.372638 57518 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0416 00:50:48.515744 57518 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0416 00:50:48.516005 57518 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
I0416 00:50:48.584821 57518 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0416 00:50:48.585196 57518 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
I0416 00:50:48.713197 57518 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0416 00:50:48.981933 57518 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0416 00:50:49.179574 57518 kubeadm.go:309] [certs] Generating "sa" key and public key
I0416 00:50:49.179835 57518 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0416 00:50:49.615009 57518 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0416 00:50:49.788794 57518 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0416 00:50:49.912864 57518 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0416 00:50:50.137771 57518 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0416 00:50:50.156931 57518 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0416 00:50:50.158435 57518 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0416 00:50:50.158497 57518 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0416 00:50:50.302670 57518 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0416 00:50:50.304630 57518 out.go:204] - Booting up control plane ...
I0416 00:50:50.304770 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0416 00:50:50.316404 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0416 00:50:50.317255 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0416 00:50:50.318261 57518 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0416 00:50:50.324142 57518 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0416 00:51:30.310285 57518 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0416 00:51:30.310384 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:51:30.310573 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:51:35.310173 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:51:35.310362 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:51:45.309654 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:51:45.309909 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:52:05.309822 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:52:05.310111 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:52:45.311106 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:52:45.311704 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:52:45.311730 57518 kubeadm.go:309]
I0416 00:52:45.311846 57518 kubeadm.go:309] Unfortunately, an error has occurred:
I0416 00:52:45.311944 57518 kubeadm.go:309] timed out waiting for the condition
I0416 00:52:45.311954 57518 kubeadm.go:309]
I0416 00:52:45.312032 57518 kubeadm.go:309] This error is likely caused by:
I0416 00:52:45.312127 57518 kubeadm.go:309] - The kubelet is not running
I0416 00:52:45.312360 57518 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0416 00:52:45.312371 57518 kubeadm.go:309]
I0416 00:52:45.312614 57518 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0416 00:52:45.312696 57518 kubeadm.go:309] - 'systemctl status kubelet'
I0416 00:52:45.312770 57518 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0416 00:52:45.312779 57518 kubeadm.go:309]
I0416 00:52:45.313048 57518 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0416 00:52:45.313291 57518 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0416 00:52:45.313315 57518 kubeadm.go:309]
I0416 00:52:45.313512 57518 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0416 00:52:45.313662 57518 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0416 00:52:45.313856 57518 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0416 00:52:45.314033 57518 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0416 00:52:45.314051 57518 kubeadm.go:309]
I0416 00:52:45.314244 57518 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0416 00:52:45.314580 57518 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0416 00:52:45.314722 57518 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
W0416 00:52:45.315122 57518 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-800769] and IPs [192.168.83.98 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0416 00:52:45.315189 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0416 00:52:46.474891 57518 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force": (1.159681728s)
I0416 00:52:46.474951 57518 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0416 00:52:46.489446 57518 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0416 00:52:46.500558 57518 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0416 00:52:46.500577 57518 kubeadm.go:156] found existing configuration files:
I0416 00:52:46.500625 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0416 00:52:46.510206 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0416 00:52:46.510250 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0416 00:52:46.519942 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0416 00:52:46.529541 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0416 00:52:46.529592 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0416 00:52:46.539459 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0416 00:52:46.548955 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0416 00:52:46.549008 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0416 00:52:46.558758 57518 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0416 00:52:46.568041 57518 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0416 00:52:46.568093 57518 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0416 00:52:46.577904 57518 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0416 00:52:46.797114 57518 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0416 00:54:43.007831 57518 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0416 00:54:43.007920 57518 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0416 00:54:43.009467 57518 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0416 00:54:43.009534 57518 kubeadm.go:309] [preflight] Running pre-flight checks
I0416 00:54:43.009613 57518 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0416 00:54:43.009724 57518 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0416 00:54:43.009814 57518 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0416 00:54:43.009891 57518 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0416 00:54:43.012089 57518 out.go:204] - Generating certificates and keys ...
I0416 00:54:43.012149 57518 kubeadm.go:309] [certs] Using existing ca certificate authority
I0416 00:54:43.012204 57518 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0416 00:54:43.012291 57518 kubeadm.go:309] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0416 00:54:43.012382 57518 kubeadm.go:309] [certs] Using existing front-proxy-ca certificate authority
I0416 00:54:43.012540 57518 kubeadm.go:309] [certs] Using existing front-proxy-client certificate and key on disk
I0416 00:54:43.012619 57518 kubeadm.go:309] [certs] Using existing etcd/ca certificate authority
I0416 00:54:43.012705 57518 kubeadm.go:309] [certs] Using existing etcd/server certificate and key on disk
I0416 00:54:43.012804 57518 kubeadm.go:309] [certs] Using existing etcd/peer certificate and key on disk
I0416 00:54:43.012879 57518 kubeadm.go:309] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0416 00:54:43.012970 57518 kubeadm.go:309] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0416 00:54:43.013009 57518 kubeadm.go:309] [certs] Using the existing "sa" key
I0416 00:54:43.013090 57518 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0416 00:54:43.013152 57518 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0416 00:54:43.013246 57518 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0416 00:54:43.013342 57518 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0416 00:54:43.013428 57518 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0416 00:54:43.013532 57518 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0416 00:54:43.013605 57518 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0416 00:54:43.013652 57518 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0416 00:54:43.013738 57518 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0416 00:54:43.015440 57518 out.go:204] - Booting up control plane ...
I0416 00:54:43.015545 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0416 00:54:43.015613 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0416 00:54:43.015699 57518 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0416 00:54:43.015802 57518 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0416 00:54:43.016005 57518 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0416 00:54:43.016055 57518 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0416 00:54:43.016116 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:54:43.016277 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:54:43.016336 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:54:43.016511 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:54:43.016570 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:54:43.016727 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:54:43.016788 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:54:43.016988 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:54:43.017053 57518 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0416 00:54:43.017247 57518 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0416 00:54:43.017258 57518 kubeadm.go:309]
I0416 00:54:43.017295 57518 kubeadm.go:309] Unfortunately, an error has occurred:
I0416 00:54:43.017341 57518 kubeadm.go:309] timed out waiting for the condition
I0416 00:54:43.017348 57518 kubeadm.go:309]
I0416 00:54:43.017376 57518 kubeadm.go:309] This error is likely caused by:
I0416 00:54:43.017405 57518 kubeadm.go:309] - The kubelet is not running
I0416 00:54:43.017495 57518 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0416 00:54:43.017504 57518 kubeadm.go:309]
I0416 00:54:43.017622 57518 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0416 00:54:43.017674 57518 kubeadm.go:309] - 'systemctl status kubelet'
I0416 00:54:43.017716 57518 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0416 00:54:43.017725 57518 kubeadm.go:309]
I0416 00:54:43.017873 57518 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0416 00:54:43.017991 57518 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0416 00:54:43.018001 57518 kubeadm.go:309]
I0416 00:54:43.018098 57518 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0416 00:54:43.018192 57518 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0416 00:54:43.018294 57518 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0416 00:54:43.018390 57518 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0416 00:54:43.018413 57518 kubeadm.go:309]
I0416 00:54:43.018456 57518 kubeadm.go:393] duration metric: took 3m55.969423615s to StartCluster
I0416 00:54:43.018494 57518 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0416 00:54:43.018559 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0416 00:54:43.061584 57518 cri.go:89] found id: ""
I0416 00:54:43.061613 57518 logs.go:276] 0 containers: []
W0416 00:54:43.061624 57518 logs.go:278] No container was found matching "kube-apiserver"
I0416 00:54:43.061631 57518 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0416 00:54:43.061693 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0416 00:54:43.097755 57518 cri.go:89] found id: ""
I0416 00:54:43.097776 57518 logs.go:276] 0 containers: []
W0416 00:54:43.097783 57518 logs.go:278] No container was found matching "etcd"
I0416 00:54:43.097789 57518 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0416 00:54:43.097842 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0416 00:54:43.133910 57518 cri.go:89] found id: ""
I0416 00:54:43.133931 57518 logs.go:276] 0 containers: []
W0416 00:54:43.133938 57518 logs.go:278] No container was found matching "coredns"
I0416 00:54:43.133947 57518 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0416 00:54:43.134005 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0416 00:54:43.168844 57518 cri.go:89] found id: ""
I0416 00:54:43.168868 57518 logs.go:276] 0 containers: []
W0416 00:54:43.168875 57518 logs.go:278] No container was found matching "kube-scheduler"
I0416 00:54:43.168881 57518 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0416 00:54:43.168924 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0416 00:54:43.202906 57518 cri.go:89] found id: ""
I0416 00:54:43.202940 57518 logs.go:276] 0 containers: []
W0416 00:54:43.202948 57518 logs.go:278] No container was found matching "kube-proxy"
I0416 00:54:43.202954 57518 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0416 00:54:43.203003 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0416 00:54:43.237722 57518 cri.go:89] found id: ""
I0416 00:54:43.237746 57518 logs.go:276] 0 containers: []
W0416 00:54:43.237754 57518 logs.go:278] No container was found matching "kube-controller-manager"
I0416 00:54:43.237759 57518 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0416 00:54:43.237814 57518 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0416 00:54:43.272696 57518 cri.go:89] found id: ""
I0416 00:54:43.272725 57518 logs.go:276] 0 containers: []
W0416 00:54:43.272735 57518 logs.go:278] No container was found matching "kindnet"
I0416 00:54:43.272746 57518 logs.go:123] Gathering logs for container status ...
I0416 00:54:43.272759 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0416 00:54:43.310592 57518 logs.go:123] Gathering logs for kubelet ...
I0416 00:54:43.310620 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0416 00:54:43.360454 57518 logs.go:123] Gathering logs for dmesg ...
I0416 00:54:43.360488 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0416 00:54:43.374688 57518 logs.go:123] Gathering logs for describe nodes ...
I0416 00:54:43.374716 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0416 00:54:43.490351 57518 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0416 00:54:43.490373 57518 logs.go:123] Gathering logs for CRI-O ...
I0416 00:54:43.490384 57518 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
W0416 00:54:43.599016 57518 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0416 00:54:43.599081 57518 out.go:239] *
*
W0416 00:54:43.599148 57518 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0416 00:54:43.599180 57518 out.go:239] *
*
W0416 00:54:43.599980 57518 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0416 00:54:43.603216 57518 out.go:177]
W0416 00:54:43.604651 57518 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0416 00:54:43.604705 57518 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0416 00:54:43.604734 57518 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0416 00:54:43.606438 57518 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-800769 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-800769 -n old-k8s-version-800769
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-800769 -n old-k8s-version-800769: exit status 6 (241.427011ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0416 00:54:43.893672 61319 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-800769" does not appear in /home/jenkins/minikube-integration/18647-7542/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-800769" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (269.22s)