=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-431084 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-431084 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m45.163712235s)
-- stdout --
* [old-k8s-version-431084] minikube v1.34.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19640
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19640-5422/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19640-5422/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-431084" primary control-plane node in "old-k8s-version-431084" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0914 00:52:35.724587 66801 out.go:345] Setting OutFile to fd 1 ...
I0914 00:52:35.724870 66801 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0914 00:52:35.724882 66801 out.go:358] Setting ErrFile to fd 2...
I0914 00:52:35.724887 66801 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0914 00:52:35.725072 66801 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19640-5422/.minikube/bin
I0914 00:52:35.725683 66801 out.go:352] Setting JSON to false
I0914 00:52:35.726845 66801 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":5702,"bootTime":1726269454,"procs":304,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1068-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0914 00:52:35.726957 66801 start.go:139] virtualization: kvm guest
I0914 00:52:35.729922 66801 out.go:177] * [old-k8s-version-431084] minikube v1.34.0 on Ubuntu 20.04 (kvm/amd64)
I0914 00:52:35.731826 66801 out.go:177] - MINIKUBE_LOCATION=19640
I0914 00:52:35.731857 66801 notify.go:220] Checking for updates...
I0914 00:52:35.735497 66801 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0914 00:52:35.737400 66801 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19640-5422/kubeconfig
I0914 00:52:35.738944 66801 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19640-5422/.minikube
I0914 00:52:35.740627 66801 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0914 00:52:35.742223 66801 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0914 00:52:35.744593 66801 config.go:182] Loaded profile config "bridge-670449": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.1
I0914 00:52:35.744763 66801 config.go:182] Loaded profile config "flannel-670449": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.1
I0914 00:52:35.744950 66801 config.go:182] Loaded profile config "pause-609507": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.1
I0914 00:52:35.745082 66801 driver.go:394] Setting default libvirt URI to qemu:///system
I0914 00:52:35.792655 66801 out.go:177] * Using the kvm2 driver based on user configuration
I0914 00:52:35.794325 66801 start.go:297] selected driver: kvm2
I0914 00:52:35.794345 66801 start.go:901] validating driver "kvm2" against <nil>
I0914 00:52:35.794357 66801 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0914 00:52:35.795353 66801 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0914 00:52:35.795460 66801 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19640-5422/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0914 00:52:35.812779 66801 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.34.0
I0914 00:52:35.812843 66801 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0914 00:52:35.813119 66801 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0914 00:52:35.813151 66801 cni.go:84] Creating CNI manager for ""
I0914 00:52:35.813197 66801 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0914 00:52:35.813206 66801 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0914 00:52:35.813298 66801 start.go:340] cluster config:
{Name:old-k8s-version-431084 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726243947-19640@sha256:bb1287c9c0ec51ba7d8272f0f8073d6e9758ad79ff87c787fdce1c3513743243 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-431084 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0914 00:52:35.813422 66801 iso.go:125] acquiring lock: {Name:mk847265edcfde6c31d5f8c0ad489f5e3ef3fe23 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0914 00:52:35.815527 66801 out.go:177] * Starting "old-k8s-version-431084" primary control-plane node in "old-k8s-version-431084" cluster
I0914 00:52:35.816967 66801 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0914 00:52:35.817022 66801 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0914 00:52:35.817033 66801 cache.go:56] Caching tarball of preloaded images
I0914 00:52:35.817165 66801 preload.go:172] Found /home/jenkins/minikube-integration/19640-5422/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0914 00:52:35.817181 66801 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0914 00:52:35.817348 66801 profile.go:143] Saving config to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/config.json ...
I0914 00:52:35.817378 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/config.json: {Name:mk66cd4353dae42258dd8e2fe6f383f65dc09589 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:52:35.817576 66801 start.go:360] acquireMachinesLock for old-k8s-version-431084: {Name:mk217f2566ba17084c34ea86e690e58b0176e948 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0914 00:52:51.328508 66801 start.go:364] duration metric: took 15.510868976s to acquireMachinesLock for "old-k8s-version-431084"
I0914 00:52:51.328574 66801 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-431084 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19640/minikube-v1.34.0-1726243933-19640-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726243947-19640@sha256:bb1287c9c0ec51ba7d8272f0f8073d6e9758ad79ff87c787fdce1c3513743243 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-431084 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0914 00:52:51.328690 66801 start.go:125] createHost starting for "" (driver="kvm2")
I0914 00:52:51.330804 66801 out.go:235] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0914 00:52:51.331044 66801 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0914 00:52:51.331098 66801 main.go:141] libmachine: Launching plugin server for driver kvm2
I0914 00:52:51.348179 66801 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:33873
I0914 00:52:51.348690 66801 main.go:141] libmachine: () Calling .GetVersion
I0914 00:52:51.349350 66801 main.go:141] libmachine: Using API Version 1
I0914 00:52:51.349375 66801 main.go:141] libmachine: () Calling .SetConfigRaw
I0914 00:52:51.349795 66801 main.go:141] libmachine: () Calling .GetMachineName
I0914 00:52:51.349981 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetMachineName
I0914 00:52:51.350148 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:52:51.350313 66801 start.go:159] libmachine.API.Create for "old-k8s-version-431084" (driver="kvm2")
I0914 00:52:51.350346 66801 client.go:168] LocalClient.Create starting
I0914 00:52:51.350381 66801 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem
I0914 00:52:51.350426 66801 main.go:141] libmachine: Decoding PEM data...
I0914 00:52:51.350450 66801 main.go:141] libmachine: Parsing certificate...
I0914 00:52:51.350517 66801 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19640-5422/.minikube/certs/cert.pem
I0914 00:52:51.350545 66801 main.go:141] libmachine: Decoding PEM data...
I0914 00:52:51.350565 66801 main.go:141] libmachine: Parsing certificate...
I0914 00:52:51.350590 66801 main.go:141] libmachine: Running pre-create checks...
I0914 00:52:51.350607 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .PreCreateCheck
I0914 00:52:51.350931 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetConfigRaw
I0914 00:52:51.351327 66801 main.go:141] libmachine: Creating machine...
I0914 00:52:51.351341 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .Create
I0914 00:52:51.351507 66801 main.go:141] libmachine: (old-k8s-version-431084) Creating KVM machine...
I0914 00:52:51.352625 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found existing default KVM network
I0914 00:52:51.353662 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.353505 66991 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr3 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:44:cd:68} reservation:<nil>}
I0914 00:52:51.354641 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.354562 66991 network.go:211] skipping subnet 192.168.50.0/24 that is taken: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.50.1 IfaceMTU:1500 IfaceMAC:52:54:00:ce:96:2b} reservation:<nil>}
I0914 00:52:51.355823 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.355718 66991 network.go:206] using free private subnet 192.168.61.0/24: &{IP:192.168.61.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.61.0/24 Gateway:192.168.61.1 ClientMin:192.168.61.2 ClientMax:192.168.61.254 Broadcast:192.168.61.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000380930}
I0914 00:52:51.355857 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | created network xml:
I0914 00:52:51.355869 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <network>
I0914 00:52:51.355875 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <name>mk-old-k8s-version-431084</name>
I0914 00:52:51.355891 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <dns enable='no'/>
I0914 00:52:51.355895 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG |
I0914 00:52:51.355902 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <ip address='192.168.61.1' netmask='255.255.255.0'>
I0914 00:52:51.355907 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <dhcp>
I0914 00:52:51.355916 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | <range start='192.168.61.2' end='192.168.61.253'/>
I0914 00:52:51.355920 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | </dhcp>
I0914 00:52:51.355925 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | </ip>
I0914 00:52:51.355930 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG |
I0914 00:52:51.355937 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | </network>
I0914 00:52:51.355944 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG |
I0914 00:52:51.364017 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | trying to create private KVM network mk-old-k8s-version-431084 192.168.61.0/24...
I0914 00:52:51.440773 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting up store path in /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084 ...
I0914 00:52:51.440805 66801 main.go:141] libmachine: (old-k8s-version-431084) Building disk image from file:///home/jenkins/minikube-integration/19640-5422/.minikube/cache/iso/amd64/minikube-v1.34.0-1726243933-19640-amd64.iso
I0914 00:52:51.440818 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | private KVM network mk-old-k8s-version-431084 192.168.61.0/24 created
I0914 00:52:51.440831 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.440744 66991 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19640-5422/.minikube
I0914 00:52:51.440852 66801 main.go:141] libmachine: (old-k8s-version-431084) Downloading /home/jenkins/minikube-integration/19640-5422/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19640-5422/.minikube/cache/iso/amd64/minikube-v1.34.0-1726243933-19640-amd64.iso...
I0914 00:52:51.735078 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.734905 66991 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa...
I0914 00:52:51.899652 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.899507 66991 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/old-k8s-version-431084.rawdisk...
I0914 00:52:51.899696 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Writing magic tar header
I0914 00:52:51.899714 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Writing SSH key tar header
I0914 00:52:51.899726 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:51.899685 66991 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084 ...
I0914 00:52:51.899876 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084
I0914 00:52:51.899901 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19640-5422/.minikube/machines
I0914 00:52:51.899915 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084 (perms=drwx------)
I0914 00:52:51.899925 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19640-5422/.minikube
I0914 00:52:51.899945 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19640-5422
I0914 00:52:51.899957 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0914 00:52:51.899967 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home/jenkins
I0914 00:52:51.899975 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Checking permissions on dir: /home
I0914 00:52:51.899988 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins/minikube-integration/19640-5422/.minikube/machines (perms=drwxr-xr-x)
I0914 00:52:51.899998 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Skipping /home - not owner
I0914 00:52:51.900017 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins/minikube-integration/19640-5422/.minikube (perms=drwxr-xr-x)
I0914 00:52:51.900030 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins/minikube-integration/19640-5422 (perms=drwxrwxr-x)
I0914 00:52:51.900057 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0914 00:52:51.900075 66801 main.go:141] libmachine: (old-k8s-version-431084) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0914 00:52:51.900089 66801 main.go:141] libmachine: (old-k8s-version-431084) Creating domain...
I0914 00:52:51.901527 66801 main.go:141] libmachine: (old-k8s-version-431084) define libvirt domain using xml:
I0914 00:52:51.901552 66801 main.go:141] libmachine: (old-k8s-version-431084) <domain type='kvm'>
I0914 00:52:51.901574 66801 main.go:141] libmachine: (old-k8s-version-431084) <name>old-k8s-version-431084</name>
I0914 00:52:51.901605 66801 main.go:141] libmachine: (old-k8s-version-431084) <memory unit='MiB'>2200</memory>
I0914 00:52:51.901613 66801 main.go:141] libmachine: (old-k8s-version-431084) <vcpu>2</vcpu>
I0914 00:52:51.901626 66801 main.go:141] libmachine: (old-k8s-version-431084) <features>
I0914 00:52:51.901633 66801 main.go:141] libmachine: (old-k8s-version-431084) <acpi/>
I0914 00:52:51.901644 66801 main.go:141] libmachine: (old-k8s-version-431084) <apic/>
I0914 00:52:51.901649 66801 main.go:141] libmachine: (old-k8s-version-431084) <pae/>
I0914 00:52:51.901656 66801 main.go:141] libmachine: (old-k8s-version-431084)
I0914 00:52:51.901675 66801 main.go:141] libmachine: (old-k8s-version-431084) </features>
I0914 00:52:51.901691 66801 main.go:141] libmachine: (old-k8s-version-431084) <cpu mode='host-passthrough'>
I0914 00:52:51.901731 66801 main.go:141] libmachine: (old-k8s-version-431084)
I0914 00:52:51.901751 66801 main.go:141] libmachine: (old-k8s-version-431084) </cpu>
I0914 00:52:51.901761 66801 main.go:141] libmachine: (old-k8s-version-431084) <os>
I0914 00:52:51.901771 66801 main.go:141] libmachine: (old-k8s-version-431084) <type>hvm</type>
I0914 00:52:51.901780 66801 main.go:141] libmachine: (old-k8s-version-431084) <boot dev='cdrom'/>
I0914 00:52:51.901786 66801 main.go:141] libmachine: (old-k8s-version-431084) <boot dev='hd'/>
I0914 00:52:51.901795 66801 main.go:141] libmachine: (old-k8s-version-431084) <bootmenu enable='no'/>
I0914 00:52:51.901800 66801 main.go:141] libmachine: (old-k8s-version-431084) </os>
I0914 00:52:51.901809 66801 main.go:141] libmachine: (old-k8s-version-431084) <devices>
I0914 00:52:51.901816 66801 main.go:141] libmachine: (old-k8s-version-431084) <disk type='file' device='cdrom'>
I0914 00:52:51.901829 66801 main.go:141] libmachine: (old-k8s-version-431084) <source file='/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/boot2docker.iso'/>
I0914 00:52:51.901836 66801 main.go:141] libmachine: (old-k8s-version-431084) <target dev='hdc' bus='scsi'/>
I0914 00:52:51.901857 66801 main.go:141] libmachine: (old-k8s-version-431084) <readonly/>
I0914 00:52:51.901867 66801 main.go:141] libmachine: (old-k8s-version-431084) </disk>
I0914 00:52:51.901878 66801 main.go:141] libmachine: (old-k8s-version-431084) <disk type='file' device='disk'>
I0914 00:52:51.901892 66801 main.go:141] libmachine: (old-k8s-version-431084) <driver name='qemu' type='raw' cache='default' io='threads' />
I0914 00:52:51.901909 66801 main.go:141] libmachine: (old-k8s-version-431084) <source file='/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/old-k8s-version-431084.rawdisk'/>
I0914 00:52:51.901920 66801 main.go:141] libmachine: (old-k8s-version-431084) <target dev='hda' bus='virtio'/>
I0914 00:52:51.901931 66801 main.go:141] libmachine: (old-k8s-version-431084) </disk>
I0914 00:52:51.901943 66801 main.go:141] libmachine: (old-k8s-version-431084) <interface type='network'>
I0914 00:52:51.901957 66801 main.go:141] libmachine: (old-k8s-version-431084) <source network='mk-old-k8s-version-431084'/>
I0914 00:52:51.901966 66801 main.go:141] libmachine: (old-k8s-version-431084) <model type='virtio'/>
I0914 00:52:51.901975 66801 main.go:141] libmachine: (old-k8s-version-431084) </interface>
I0914 00:52:51.901982 66801 main.go:141] libmachine: (old-k8s-version-431084) <interface type='network'>
I0914 00:52:51.901994 66801 main.go:141] libmachine: (old-k8s-version-431084) <source network='default'/>
I0914 00:52:51.902000 66801 main.go:141] libmachine: (old-k8s-version-431084) <model type='virtio'/>
I0914 00:52:51.902010 66801 main.go:141] libmachine: (old-k8s-version-431084) </interface>
I0914 00:52:51.902021 66801 main.go:141] libmachine: (old-k8s-version-431084) <serial type='pty'>
I0914 00:52:51.902033 66801 main.go:141] libmachine: (old-k8s-version-431084) <target port='0'/>
I0914 00:52:51.902040 66801 main.go:141] libmachine: (old-k8s-version-431084) </serial>
I0914 00:52:51.902052 66801 main.go:141] libmachine: (old-k8s-version-431084) <console type='pty'>
I0914 00:52:51.902062 66801 main.go:141] libmachine: (old-k8s-version-431084) <target type='serial' port='0'/>
I0914 00:52:51.902072 66801 main.go:141] libmachine: (old-k8s-version-431084) </console>
I0914 00:52:51.902081 66801 main.go:141] libmachine: (old-k8s-version-431084) <rng model='virtio'>
I0914 00:52:51.902091 66801 main.go:141] libmachine: (old-k8s-version-431084) <backend model='random'>/dev/random</backend>
I0914 00:52:51.902100 66801 main.go:141] libmachine: (old-k8s-version-431084) </rng>
I0914 00:52:51.902107 66801 main.go:141] libmachine: (old-k8s-version-431084)
I0914 00:52:51.902116 66801 main.go:141] libmachine: (old-k8s-version-431084)
I0914 00:52:51.902133 66801 main.go:141] libmachine: (old-k8s-version-431084) </devices>
I0914 00:52:51.902144 66801 main.go:141] libmachine: (old-k8s-version-431084) </domain>
I0914 00:52:51.902155 66801 main.go:141] libmachine: (old-k8s-version-431084)
I0914 00:52:51.906817 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:63:e1:fc in network default
I0914 00:52:51.907735 66801 main.go:141] libmachine: (old-k8s-version-431084) Ensuring networks are active...
I0914 00:52:51.907769 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:51.908690 66801 main.go:141] libmachine: (old-k8s-version-431084) Ensuring network default is active
I0914 00:52:51.909010 66801 main.go:141] libmachine: (old-k8s-version-431084) Ensuring network mk-old-k8s-version-431084 is active
I0914 00:52:51.909570 66801 main.go:141] libmachine: (old-k8s-version-431084) Getting domain xml...
I0914 00:52:51.910517 66801 main.go:141] libmachine: (old-k8s-version-431084) Creating domain...
I0914 00:52:53.472296 66801 main.go:141] libmachine: (old-k8s-version-431084) Waiting to get IP...
I0914 00:52:53.473458 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:53.474119 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:53.474172 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:53.474109 66991 retry.go:31] will retry after 277.653713ms: waiting for machine to come up
I0914 00:52:53.753876 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:53.755354 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:53.755382 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:53.755255 66991 retry.go:31] will retry after 372.557708ms: waiting for machine to come up
I0914 00:52:54.129933 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:54.130551 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:54.130578 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:54.130504 66991 retry.go:31] will retry after 329.217104ms: waiting for machine to come up
I0914 00:52:54.461115 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:54.461742 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:54.461767 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:54.461660 66991 retry.go:31] will retry after 534.468325ms: waiting for machine to come up
I0914 00:52:54.998338 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:54.999189 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:54.999215 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:54.999096 66991 retry.go:31] will retry after 529.424126ms: waiting for machine to come up
I0914 00:52:55.529670 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:55.530157 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:55.530193 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:55.530103 66991 retry.go:31] will retry after 701.848536ms: waiting for machine to come up
I0914 00:52:56.234175 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:56.234644 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:56.234675 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:56.234584 66991 retry.go:31] will retry after 926.106578ms: waiting for machine to come up
I0914 00:52:57.162172 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:57.162686 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:57.162715 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:57.162647 66991 retry.go:31] will retry after 1.270446243s: waiting for machine to come up
I0914 00:52:58.435104 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:58.435636 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:58.435665 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:58.435587 66991 retry.go:31] will retry after 1.16744392s: waiting for machine to come up
I0914 00:52:59.604970 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:52:59.605514 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:52:59.605541 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:52:59.605457 66991 retry.go:31] will retry after 1.768720127s: waiting for machine to come up
I0914 00:53:01.375890 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:01.376460 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:53:01.376502 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:53:01.376418 66991 retry.go:31] will retry after 2.152913439s: waiting for machine to come up
I0914 00:53:03.530645 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:03.531243 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:53:03.531267 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:53:03.531195 66991 retry.go:31] will retry after 2.194352636s: waiting for machine to come up
I0914 00:53:05.728371 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:05.728822 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:53:05.728843 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:53:05.728779 66991 retry.go:31] will retry after 3.501013157s: waiting for machine to come up
I0914 00:53:09.231390 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:09.232039 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find current IP address of domain old-k8s-version-431084 in network mk-old-k8s-version-431084
I0914 00:53:09.232061 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | I0914 00:53:09.231992 66991 retry.go:31] will retry after 4.974590479s: waiting for machine to come up
I0914 00:53:14.209543 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.210118 66801 main.go:141] libmachine: (old-k8s-version-431084) Found IP for machine: 192.168.61.116
I0914 00:53:14.210146 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has current primary IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.210154 66801 main.go:141] libmachine: (old-k8s-version-431084) Reserving static IP address...
I0914 00:53:14.210593 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-431084", mac: "52:54:00:d9:88:87", ip: "192.168.61.116"} in network mk-old-k8s-version-431084
I0914 00:53:14.296971 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Getting to WaitForSSH function...
I0914 00:53:14.296997 66801 main.go:141] libmachine: (old-k8s-version-431084) Reserved static IP address: 192.168.61.116
I0914 00:53:14.297027 66801 main.go:141] libmachine: (old-k8s-version-431084) Waiting for SSH to be available...
I0914 00:53:14.302432 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.302836 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:minikube Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.302946 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.303093 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Using SSH client type: external
I0914 00:53:14.303124 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Using SSH private key: /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa (-rw-------)
I0914 00:53:14.303154 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.61.116 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa -p 22] /usr/bin/ssh <nil>}
I0914 00:53:14.303167 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | About to run SSH command:
I0914 00:53:14.303184 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | exit 0
I0914 00:53:14.431746 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | SSH cmd err, output: <nil>:
I0914 00:53:14.432030 66801 main.go:141] libmachine: (old-k8s-version-431084) KVM machine creation complete!
I0914 00:53:14.432386 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetConfigRaw
I0914 00:53:14.432983 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:14.433194 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:14.433352 66801 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0914 00:53:14.433368 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetState
I0914 00:53:14.434702 66801 main.go:141] libmachine: Detecting operating system of created instance...
I0914 00:53:14.434718 66801 main.go:141] libmachine: Waiting for SSH to be available...
I0914 00:53:14.434725 66801 main.go:141] libmachine: Getting to WaitForSSH function...
I0914 00:53:14.434733 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:14.436931 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.437354 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.437386 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.437454 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:14.437635 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.437770 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.437923 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:14.438141 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:14.438373 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:14.438387 66801 main.go:141] libmachine: About to run SSH command:
exit 0
I0914 00:53:14.543589 66801 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0914 00:53:14.543616 66801 main.go:141] libmachine: Detecting the provisioner...
I0914 00:53:14.543627 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:14.546738 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.547137 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.547170 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.547318 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:14.547498 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.547675 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.547822 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:14.547957 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:14.548166 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:14.548178 66801 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0914 00:53:14.649639 66801 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0914 00:53:14.649735 66801 main.go:141] libmachine: found compatible host: buildroot
I0914 00:53:14.649745 66801 main.go:141] libmachine: Provisioning with buildroot...
I0914 00:53:14.649755 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetMachineName
I0914 00:53:14.650015 66801 buildroot.go:166] provisioning hostname "old-k8s-version-431084"
I0914 00:53:14.650053 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetMachineName
I0914 00:53:14.650225 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:14.653638 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.654117 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.654141 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.654349 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:14.654544 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.654724 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.654888 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:14.655069 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:14.655291 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:14.655303 66801 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-431084 && echo "old-k8s-version-431084" | sudo tee /etc/hostname
I0914 00:53:14.775286 66801 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-431084
I0914 00:53:14.775327 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:14.778779 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.779250 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.779282 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.779593 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:14.779844 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.780078 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:14.780275 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:14.780542 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:14.780766 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:14.780793 66801 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-431084' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-431084/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-431084' | sudo tee -a /etc/hosts;
fi
fi
I0914 00:53:14.901057 66801 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0914 00:53:14.901087 66801 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19640-5422/.minikube CaCertPath:/home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19640-5422/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19640-5422/.minikube}
I0914 00:53:14.901110 66801 buildroot.go:174] setting up certificates
I0914 00:53:14.901122 66801 provision.go:84] configureAuth start
I0914 00:53:14.901132 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetMachineName
I0914 00:53:14.901438 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetIP
I0914 00:53:14.904596 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.905161 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.905192 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.905360 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:14.907972 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.908410 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:14.908440 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:14.908581 66801 provision.go:143] copyHostCerts
I0914 00:53:14.908639 66801 exec_runner.go:144] found /home/jenkins/minikube-integration/19640-5422/.minikube/key.pem, removing ...
I0914 00:53:14.908649 66801 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19640-5422/.minikube/key.pem
I0914 00:53:14.908717 66801 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19640-5422/.minikube/key.pem (1679 bytes)
I0914 00:53:14.908801 66801 exec_runner.go:144] found /home/jenkins/minikube-integration/19640-5422/.minikube/ca.pem, removing ...
I0914 00:53:14.908811 66801 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19640-5422/.minikube/ca.pem
I0914 00:53:14.908841 66801 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19640-5422/.minikube/ca.pem (1078 bytes)
I0914 00:53:14.908916 66801 exec_runner.go:144] found /home/jenkins/minikube-integration/19640-5422/.minikube/cert.pem, removing ...
I0914 00:53:14.908926 66801 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19640-5422/.minikube/cert.pem
I0914 00:53:14.908958 66801 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19640-5422/.minikube/cert.pem (1123 bytes)
I0914 00:53:14.909027 66801 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19640-5422/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-431084 san=[127.0.0.1 192.168.61.116 localhost minikube old-k8s-version-431084]
I0914 00:53:15.268390 66801 provision.go:177] copyRemoteCerts
I0914 00:53:15.268453 66801 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0914 00:53:15.268476 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.271577 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.272037 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.272068 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.272265 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.272502 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.272693 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.272826 66801 sshutil.go:53] new ssh client: &{IP:192.168.61.116 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa Username:docker}
I0914 00:53:15.353915 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0914 00:53:15.378403 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0914 00:53:15.404566 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0914 00:53:15.434293 66801 provision.go:87] duration metric: took 533.159761ms to configureAuth
I0914 00:53:15.434323 66801 buildroot.go:189] setting minikube options for container-runtime
I0914 00:53:15.434495 66801 config.go:182] Loaded profile config "old-k8s-version-431084": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0914 00:53:15.434609 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.437445 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.437919 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.437943 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.438119 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.438318 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.438489 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.438658 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.438839 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:15.439095 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:15.439112 66801 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0914 00:53:15.679700 66801 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0914 00:53:15.679730 66801 main.go:141] libmachine: Checking connection to Docker...
I0914 00:53:15.679738 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetURL
I0914 00:53:15.680936 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | Using libvirt version 6000000
I0914 00:53:15.683775 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.684363 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.684396 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.684581 66801 main.go:141] libmachine: Docker is up and running!
I0914 00:53:15.684597 66801 main.go:141] libmachine: Reticulating splines...
I0914 00:53:15.684632 66801 client.go:171] duration metric: took 24.334249123s to LocalClient.Create
I0914 00:53:15.684663 66801 start.go:167] duration metric: took 24.334349462s to libmachine.API.Create "old-k8s-version-431084"
I0914 00:53:15.684675 66801 start.go:293] postStartSetup for "old-k8s-version-431084" (driver="kvm2")
I0914 00:53:15.684691 66801 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0914 00:53:15.684716 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:15.684975 66801 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0914 00:53:15.684997 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.687573 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.688050 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.688079 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.688236 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.688404 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.688524 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.688623 66801 sshutil.go:53] new ssh client: &{IP:192.168.61.116 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa Username:docker}
I0914 00:53:15.772846 66801 ssh_runner.go:195] Run: cat /etc/os-release
I0914 00:53:15.778340 66801 info.go:137] Remote host: Buildroot 2023.02.9
I0914 00:53:15.778364 66801 filesync.go:126] Scanning /home/jenkins/minikube-integration/19640-5422/.minikube/addons for local assets ...
I0914 00:53:15.778428 66801 filesync.go:126] Scanning /home/jenkins/minikube-integration/19640-5422/.minikube/files for local assets ...
I0914 00:53:15.778535 66801 filesync.go:149] local asset: /home/jenkins/minikube-integration/19640-5422/.minikube/files/etc/ssl/certs/126022.pem -> 126022.pem in /etc/ssl/certs
I0914 00:53:15.778664 66801 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0914 00:53:15.791458 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/files/etc/ssl/certs/126022.pem --> /etc/ssl/certs/126022.pem (1708 bytes)
I0914 00:53:15.825391 66801 start.go:296] duration metric: took 140.700059ms for postStartSetup
I0914 00:53:15.825448 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetConfigRaw
I0914 00:53:15.826222 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetIP
I0914 00:53:15.829799 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.830385 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.830407 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.830843 66801 profile.go:143] Saving config to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/config.json ...
I0914 00:53:15.831111 66801 start.go:128] duration metric: took 24.502405524s to createHost
I0914 00:53:15.831137 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.834298 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.834686 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.834720 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.834972 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.835229 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.835381 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.835509 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.835830 66801 main.go:141] libmachine: Using SSH client type: native
I0914 00:53:15.836040 66801 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x86c560] 0x86f240 <nil> [] 0s} 192.168.61.116 22 <nil> <nil>}
I0914 00:53:15.836055 66801 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0914 00:53:15.944516 66801 main.go:141] libmachine: SSH cmd err, output: <nil>: 1726275195.900159706
I0914 00:53:15.944538 66801 fix.go:216] guest clock: 1726275195.900159706
I0914 00:53:15.944546 66801 fix.go:229] Guest: 2024-09-14 00:53:15.900159706 +0000 UTC Remote: 2024-09-14 00:53:15.831122568 +0000 UTC m=+40.152976203 (delta=69.037138ms)
I0914 00:53:15.944569 66801 fix.go:200] guest clock delta is within tolerance: 69.037138ms
I0914 00:53:15.944575 66801 start.go:83] releasing machines lock for "old-k8s-version-431084", held for 24.616036705s
I0914 00:53:15.944597 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:15.944866 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetIP
I0914 00:53:15.947649 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.948084 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.948128 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.948304 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:15.948809 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:15.949030 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .DriverName
I0914 00:53:15.949134 66801 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0914 00:53:15.949196 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.949228 66801 ssh_runner.go:195] Run: cat /version.json
I0914 00:53:15.949253 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHHostname
I0914 00:53:15.951945 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.952016 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.952322 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.952347 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.952376 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:15.952394 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:15.952556 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.952725 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHPort
I0914 00:53:15.952736 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.952917 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.952922 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHKeyPath
I0914 00:53:15.953076 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetSSHUsername
I0914 00:53:15.953078 66801 sshutil.go:53] new ssh client: &{IP:192.168.61.116 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa Username:docker}
I0914 00:53:15.953250 66801 sshutil.go:53] new ssh client: &{IP:192.168.61.116 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19640-5422/.minikube/machines/old-k8s-version-431084/id_rsa Username:docker}
I0914 00:53:16.028572 66801 ssh_runner.go:195] Run: systemctl --version
I0914 00:53:16.068487 66801 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0914 00:53:16.244562 66801 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0914 00:53:16.251813 66801 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0914 00:53:16.251881 66801 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0914 00:53:16.272029 66801 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0914 00:53:16.272060 66801 start.go:495] detecting cgroup driver to use...
I0914 00:53:16.272133 66801 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0914 00:53:16.290364 66801 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0914 00:53:16.306421 66801 docker.go:217] disabling cri-docker service (if available) ...
I0914 00:53:16.306490 66801 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0914 00:53:16.321586 66801 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0914 00:53:16.340840 66801 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0914 00:53:16.474617 66801 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0914 00:53:16.627816 66801 docker.go:233] disabling docker service ...
I0914 00:53:16.627890 66801 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0914 00:53:16.645746 66801 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0914 00:53:16.664121 66801 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0914 00:53:16.821046 66801 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0914 00:53:16.976125 66801 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0914 00:53:16.994901 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0914 00:53:17.021769 66801 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0914 00:53:17.021830 66801 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0914 00:53:17.035707 66801 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0914 00:53:17.035799 66801 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0914 00:53:17.050119 66801 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0914 00:53:17.079610 66801 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0914 00:53:17.097149 66801 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0914 00:53:17.114899 66801 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0914 00:53:17.128149 66801 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0914 00:53:17.128206 66801 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0914 00:53:17.143819 66801 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0914 00:53:17.155977 66801 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0914 00:53:17.295241 66801 ssh_runner.go:195] Run: sudo systemctl restart crio
I0914 00:53:17.407809 66801 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0914 00:53:17.407879 66801 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0914 00:53:17.413233 66801 start.go:563] Will wait 60s for crictl version
I0914 00:53:17.413299 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:17.417011 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0914 00:53:17.458437 66801 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0914 00:53:17.458536 66801 ssh_runner.go:195] Run: crio --version
I0914 00:53:17.493914 66801 ssh_runner.go:195] Run: crio --version
I0914 00:53:17.537043 66801 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0914 00:53:17.538264 66801 main.go:141] libmachine: (old-k8s-version-431084) Calling .GetIP
I0914 00:53:17.541556 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:17.542107 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:d9:88:87", ip: ""} in network mk-old-k8s-version-431084: {Iface:virbr2 ExpiryTime:2024-09-14 01:53:07 +0000 UTC Type:0 Mac:52:54:00:d9:88:87 Iaid: IPaddr:192.168.61.116 Prefix:24 Hostname:old-k8s-version-431084 Clientid:01:52:54:00:d9:88:87}
I0914 00:53:17.542126 66801 main.go:141] libmachine: (old-k8s-version-431084) DBG | domain old-k8s-version-431084 has defined IP address 192.168.61.116 and MAC address 52:54:00:d9:88:87 in network mk-old-k8s-version-431084
I0914 00:53:17.542374 66801 ssh_runner.go:195] Run: grep 192.168.61.1 host.minikube.internal$ /etc/hosts
I0914 00:53:17.547035 66801 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.61.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0914 00:53:17.569831 66801 kubeadm.go:883] updating cluster {Name:old-k8s-version-431084 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19640/minikube-v1.34.0-1726243933-19640-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726243947-19640@sha256:bb1287c9c0ec51ba7d8272f0f8073d6e9758ad79ff87c787fdce1c3513743243 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-431084 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.61.116 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] M
ountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0914 00:53:17.569959 66801 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0914 00:53:17.570030 66801 ssh_runner.go:195] Run: sudo crictl images --output json
I0914 00:53:17.610786 66801 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0914 00:53:17.610856 66801 ssh_runner.go:195] Run: which lz4
I0914 00:53:17.614959 66801 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0914 00:53:17.619319 66801 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0914 00:53:17.619345 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0914 00:53:19.266781 66801 crio.go:462] duration metric: took 1.651866037s to copy over tarball
I0914 00:53:19.266853 66801 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0914 00:53:22.113624 66801 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.846743817s)
I0914 00:53:22.113654 66801 crio.go:469] duration metric: took 2.84684398s to extract the tarball
I0914 00:53:22.113664 66801 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0914 00:53:22.169862 66801 ssh_runner.go:195] Run: sudo crictl images --output json
I0914 00:53:22.216996 66801 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0914 00:53:22.217019 66801 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0914 00:53:22.217087 66801 image.go:135] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.217108 66801 image.go:135] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.217123 66801 image.go:135] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.217135 66801 image.go:135] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.217114 66801 image.go:135] retrieving image: registry.k8s.io/coredns:1.7.0
I0914 00:53:22.217154 66801 image.go:135] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.217138 66801 image.go:135] retrieving image: registry.k8s.io/pause:3.2
I0914 00:53:22.217090 66801 image.go:135] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0914 00:53:22.218335 66801 image.go:178] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.218346 66801 image.go:178] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.218362 66801 image.go:178] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.218418 66801 image.go:178] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.218436 66801 image.go:178] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0914 00:53:22.218503 66801 image.go:178] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0914 00:53:22.218666 66801 image.go:178] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0914 00:53:22.219092 66801 image.go:178] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.409473 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.444983 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0914 00:53:22.449915 66801 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0914 00:53:22.449970 66801 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.450015 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.451772 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.463997 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0914 00:53:22.465074 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.479336 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.479520 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.536368 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.536446 66801 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0914 00:53:22.536483 66801 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0914 00:53:22.536512 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.576727 66801 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0914 00:53:22.576772 66801 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.576820 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.606260 66801 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0914 00:53:22.606311 66801 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0914 00:53:22.606345 66801 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0914 00:53:22.606360 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.606382 66801 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.606436 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.628611 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0914 00:53:22.628614 66801 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0914 00:53:22.628702 66801 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.628729 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.628777 66801 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0914 00:53:22.628810 66801 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.628852 66801 ssh_runner.go:195] Run: which crictl
I0914 00:53:22.637937 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.637968 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.637969 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0914 00:53:22.638025 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.737394 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0914 00:53:22.737401 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.737465 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.769414 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.769467 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0914 00:53:22.769493 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0914 00:53:22.769558 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.829296 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0914 00:53:22.842571 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0914 00:53:22.862166 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:22.912533 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0914 00:53:22.933476 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0914 00:53:22.942786 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0914 00:53:22.942798 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0914 00:53:22.985826 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0914 00:53:23.006419 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0914 00:53:23.006466 66801 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0914 00:53:23.038135 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0914 00:53:23.056930 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0914 00:53:23.057029 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0914 00:53:23.093983 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0914 00:53:23.094030 66801 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0914 00:53:23.455902 66801 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0914 00:53:23.606806 66801 cache_images.go:92] duration metric: took 1.389768051s to LoadCachedImages
W0914 00:53:23.606896 66801 out.go:270] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19640-5422/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0914 00:53:23.606915 66801 kubeadm.go:934] updating node { 192.168.61.116 8443 v1.20.0 crio true true} ...
I0914 00:53:23.607021 66801 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-431084 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.61.116
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-431084 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0914 00:53:23.607100 66801 ssh_runner.go:195] Run: crio config
I0914 00:53:23.667700 66801 cni.go:84] Creating CNI manager for ""
I0914 00:53:23.667721 66801 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0914 00:53:23.667733 66801 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0914 00:53:23.667756 66801 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.61.116 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-431084 NodeName:old-k8s-version-431084 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.61.116"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.61.116 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt St
aticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0914 00:53:23.667976 66801 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.61.116
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-431084"
kubeletExtraArgs:
node-ip: 192.168.61.116
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.61.116"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0914 00:53:23.668111 66801 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0914 00:53:23.680313 66801 binaries.go:44] Found k8s binaries, skipping transfer
I0914 00:53:23.680388 66801 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0914 00:53:23.693600 66801 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (430 bytes)
I0914 00:53:23.713803 66801 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0914 00:53:23.736024 66801 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2123 bytes)
I0914 00:53:23.755636 66801 ssh_runner.go:195] Run: grep 192.168.61.116 control-plane.minikube.internal$ /etc/hosts
I0914 00:53:23.761370 66801 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.61.116 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0914 00:53:23.776434 66801 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0914 00:53:23.944035 66801 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0914 00:53:23.966368 66801 certs.go:68] Setting up /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084 for IP: 192.168.61.116
I0914 00:53:23.966395 66801 certs.go:194] generating shared ca certs ...
I0914 00:53:23.966415 66801 certs.go:226] acquiring lock for ca certs: {Name:mk34be5dca4d5e7be79350ed50e5d1cda46dbec0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:23.966576 66801 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19640-5422/.minikube/ca.key
I0914 00:53:23.966653 66801 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19640-5422/.minikube/proxy-client-ca.key
I0914 00:53:23.966667 66801 certs.go:256] generating profile certs ...
I0914 00:53:23.966741 66801 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.key
I0914 00:53:23.966768 66801 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.crt with IP's: []
I0914 00:53:24.120159 66801 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.crt ...
I0914 00:53:24.120188 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.crt: {Name:mk49c8d6c396579b13baab22399875d785c65936 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.120387 66801 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.key ...
I0914 00:53:24.120402 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/client.key: {Name:mka2977400ee278174d4611322de85cc87aeaa73 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.120510 66801 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key.58151014
I0914 00:53:24.120528 66801 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt.58151014 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.61.116]
I0914 00:53:24.464529 66801 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt.58151014 ...
I0914 00:53:24.464564 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt.58151014: {Name:mk8a6068d95f03b12e97de0f75276866bd4c8f2f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.464773 66801 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key.58151014 ...
I0914 00:53:24.464792 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key.58151014: {Name:mkd40b36d02760ed0a7ee76aab5345f5195e5dd7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.464873 66801 certs.go:381] copying /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt.58151014 -> /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt
I0914 00:53:24.464946 66801 certs.go:385] copying /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key.58151014 -> /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key
I0914 00:53:24.465013 66801 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.key
I0914 00:53:24.465039 66801 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.crt with IP's: []
I0914 00:53:24.707592 66801 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.crt ...
I0914 00:53:24.707619 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.crt: {Name:mk1a917c26bbc8383b2b45a41879deb3cc256dc1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.707808 66801 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.key ...
I0914 00:53:24.707821 66801 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.key: {Name:mkcec9ce1695d95454cff87ef2327200147f2bfa Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0914 00:53:24.707992 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/12602.pem (1338 bytes)
W0914 00:53:24.708028 66801 certs.go:480] ignoring /home/jenkins/minikube-integration/19640-5422/.minikube/certs/12602_empty.pem, impossibly tiny 0 bytes
I0914 00:53:24.708036 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca-key.pem (1679 bytes)
I0914 00:53:24.708057 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/ca.pem (1078 bytes)
I0914 00:53:24.708081 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/cert.pem (1123 bytes)
I0914 00:53:24.708103 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/certs/key.pem (1679 bytes)
I0914 00:53:24.708139 66801 certs.go:484] found cert: /home/jenkins/minikube-integration/19640-5422/.minikube/files/etc/ssl/certs/126022.pem (1708 bytes)
I0914 00:53:24.708688 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0914 00:53:24.736982 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0914 00:53:24.763532 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0914 00:53:24.790323 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0914 00:53:24.816568 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0914 00:53:24.848280 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0914 00:53:24.880587 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0914 00:53:24.909658 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/profiles/old-k8s-version-431084/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0914 00:53:24.943127 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/files/etc/ssl/certs/126022.pem --> /usr/share/ca-certificates/126022.pem (1708 bytes)
I0914 00:53:24.976203 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0914 00:53:25.003887 66801 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19640-5422/.minikube/certs/12602.pem --> /usr/share/ca-certificates/12602.pem (1338 bytes)
I0914 00:53:25.035242 66801 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0914 00:53:25.055077 66801 ssh_runner.go:195] Run: openssl version
I0914 00:53:25.061385 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0914 00:53:25.076435 66801 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0914 00:53:25.082261 66801 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 13 23:27 /usr/share/ca-certificates/minikubeCA.pem
I0914 00:53:25.082324 66801 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0914 00:53:25.090183 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0914 00:53:25.104791 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/12602.pem && ln -fs /usr/share/ca-certificates/12602.pem /etc/ssl/certs/12602.pem"
I0914 00:53:25.119111 66801 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/12602.pem
I0914 00:53:25.123430 66801 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 13 23:45 /usr/share/ca-certificates/12602.pem
I0914 00:53:25.123494 66801 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/12602.pem
I0914 00:53:25.129328 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/12602.pem /etc/ssl/certs/51391683.0"
I0914 00:53:25.141067 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/126022.pem && ln -fs /usr/share/ca-certificates/126022.pem /etc/ssl/certs/126022.pem"
I0914 00:53:25.152828 66801 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/126022.pem
I0914 00:53:25.157747 66801 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 13 23:45 /usr/share/ca-certificates/126022.pem
I0914 00:53:25.157862 66801 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/126022.pem
I0914 00:53:25.163642 66801 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/126022.pem /etc/ssl/certs/3ec20f2e.0"
I0914 00:53:25.175425 66801 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0914 00:53:25.179581 66801 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0914 00:53:25.179638 66801 kubeadm.go:392] StartCluster: {Name:old-k8s-version-431084 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19640/minikube-v1.34.0-1726243933-19640-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726243947-19640@sha256:bb1287c9c0ec51ba7d8272f0f8073d6e9758ad79ff87c787fdce1c3513743243 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-431084 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.61.116 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Moun
tPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0914 00:53:25.179718 66801 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0914 00:53:25.179757 66801 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0914 00:53:25.217030 66801 cri.go:89] found id: ""
I0914 00:53:25.217113 66801 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0914 00:53:25.228176 66801 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0914 00:53:25.238524 66801 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0914 00:53:25.248851 66801 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0914 00:53:25.248869 66801 kubeadm.go:157] found existing configuration files:
I0914 00:53:25.248917 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0914 00:53:25.259121 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0914 00:53:25.259183 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0914 00:53:25.269124 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0914 00:53:25.279421 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0914 00:53:25.279490 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0914 00:53:25.292114 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0914 00:53:25.301654 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0914 00:53:25.301721 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0914 00:53:25.314315 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0914 00:53:25.326606 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0914 00:53:25.326672 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0914 00:53:25.339753 66801 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0914 00:53:25.461382 66801 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0914 00:53:25.461487 66801 kubeadm.go:310] [preflight] Running pre-flight checks
I0914 00:53:25.638167 66801 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0914 00:53:25.638324 66801 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0914 00:53:25.638453 66801 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0914 00:53:25.871162 66801 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0914 00:53:25.872971 66801 out.go:235] - Generating certificates and keys ...
I0914 00:53:25.873083 66801 kubeadm.go:310] [certs] Using existing ca certificate authority
I0914 00:53:25.873172 66801 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0914 00:53:26.081705 66801 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0914 00:53:26.242468 66801 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0914 00:53:26.388827 66801 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0914 00:53:26.584326 66801 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0914 00:53:26.695148 66801 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0914 00:53:26.695318 66801 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
I0914 00:53:26.901858 66801 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0914 00:53:26.902520 66801 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
I0914 00:53:27.359477 66801 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0914 00:53:27.502106 66801 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0914 00:53:27.813722 66801 kubeadm.go:310] [certs] Generating "sa" key and public key
I0914 00:53:27.814203 66801 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0914 00:53:28.200454 66801 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0914 00:53:28.298359 66801 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0914 00:53:28.355702 66801 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0914 00:53:28.447278 66801 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0914 00:53:28.464199 66801 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0914 00:53:28.466317 66801 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0914 00:53:28.466413 66801 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0914 00:53:28.651405 66801 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0914 00:53:28.653179 66801 out.go:235] - Booting up control plane ...
I0914 00:53:28.653312 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0914 00:53:28.663498 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0914 00:53:28.664930 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0914 00:53:28.666058 66801 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0914 00:53:28.673914 66801 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0914 00:54:08.638893 66801 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0914 00:54:08.639364 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:54:08.639590 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:54:13.638946 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:54:13.639214 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:54:23.638799 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:54:23.639114 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:54:43.638821 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:54:43.639123 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:55:23.637970 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:55:23.638244 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:55:23.638267 66801 kubeadm.go:310]
I0914 00:55:23.638347 66801 kubeadm.go:310] Unfortunately, an error has occurred:
I0914 00:55:23.638418 66801 kubeadm.go:310] timed out waiting for the condition
I0914 00:55:23.638429 66801 kubeadm.go:310]
I0914 00:55:23.638472 66801 kubeadm.go:310] This error is likely caused by:
I0914 00:55:23.638526 66801 kubeadm.go:310] - The kubelet is not running
I0914 00:55:23.638679 66801 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0914 00:55:23.638692 66801 kubeadm.go:310]
I0914 00:55:23.638841 66801 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0914 00:55:23.638911 66801 kubeadm.go:310] - 'systemctl status kubelet'
I0914 00:55:23.638964 66801 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0914 00:55:23.638975 66801 kubeadm.go:310]
I0914 00:55:23.639126 66801 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0914 00:55:23.639237 66801 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0914 00:55:23.639250 66801 kubeadm.go:310]
I0914 00:55:23.639409 66801 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0914 00:55:23.639502 66801 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0914 00:55:23.639581 66801 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0914 00:55:23.639685 66801 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0914 00:55:23.639696 66801 kubeadm.go:310]
I0914 00:55:23.640119 66801 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0914 00:55:23.640327 66801 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0914 00:55:23.640411 66801 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
W0914 00:55:23.640556 66801 out.go:270] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-431084] and IPs [192.168.61.116 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0914 00:55:23.640599 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0914 00:55:24.098996 66801 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0914 00:55:24.113385 66801 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0914 00:55:24.125029 66801 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0914 00:55:24.125048 66801 kubeadm.go:157] found existing configuration files:
I0914 00:55:24.125098 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0914 00:55:24.136295 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0914 00:55:24.136388 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0914 00:55:24.147455 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0914 00:55:24.159487 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0914 00:55:24.159548 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0914 00:55:24.172162 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0914 00:55:24.183902 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0914 00:55:24.183975 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0914 00:55:24.194274 66801 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0914 00:55:24.202836 66801 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0914 00:55:24.202908 66801 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0914 00:55:24.211583 66801 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0914 00:55:24.275667 66801 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0914 00:55:24.275740 66801 kubeadm.go:310] [preflight] Running pre-flight checks
I0914 00:55:24.422355 66801 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0914 00:55:24.422473 66801 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0914 00:55:24.422587 66801 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0914 00:55:24.603282 66801 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0914 00:55:24.605160 66801 out.go:235] - Generating certificates and keys ...
I0914 00:55:24.605264 66801 kubeadm.go:310] [certs] Using existing ca certificate authority
I0914 00:55:24.605351 66801 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0914 00:55:24.605454 66801 kubeadm.go:310] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0914 00:55:24.605533 66801 kubeadm.go:310] [certs] Using existing front-proxy-ca certificate authority
I0914 00:55:24.605633 66801 kubeadm.go:310] [certs] Using existing front-proxy-client certificate and key on disk
I0914 00:55:24.605691 66801 kubeadm.go:310] [certs] Using existing etcd/ca certificate authority
I0914 00:55:24.605745 66801 kubeadm.go:310] [certs] Using existing etcd/server certificate and key on disk
I0914 00:55:24.605810 66801 kubeadm.go:310] [certs] Using existing etcd/peer certificate and key on disk
I0914 00:55:24.605906 66801 kubeadm.go:310] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0914 00:55:24.605982 66801 kubeadm.go:310] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0914 00:55:24.606017 66801 kubeadm.go:310] [certs] Using the existing "sa" key
I0914 00:55:24.606074 66801 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0914 00:55:24.772320 66801 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0914 00:55:24.860076 66801 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0914 00:55:24.914546 66801 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0914 00:55:25.061341 66801 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0914 00:55:25.087975 66801 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0914 00:55:25.088111 66801 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0914 00:55:25.088148 66801 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0914 00:55:25.239301 66801 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0914 00:55:25.241140 66801 out.go:235] - Booting up control plane ...
I0914 00:55:25.241279 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0914 00:55:25.244423 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0914 00:55:25.253381 66801 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0914 00:55:25.254389 66801 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0914 00:55:25.257450 66801 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0914 00:56:05.259917 66801 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0914 00:56:05.260042 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:56:05.260332 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:56:10.260969 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:56:10.261262 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:56:20.261794 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:56:20.262025 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:56:40.263705 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:56:40.263946 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:57:20.262706 66801 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0914 00:57:20.262973 66801 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0914 00:57:20.262988 66801 kubeadm.go:310]
I0914 00:57:20.263048 66801 kubeadm.go:310] Unfortunately, an error has occurred:
I0914 00:57:20.263104 66801 kubeadm.go:310] timed out waiting for the condition
I0914 00:57:20.263126 66801 kubeadm.go:310]
I0914 00:57:20.263167 66801 kubeadm.go:310] This error is likely caused by:
I0914 00:57:20.263207 66801 kubeadm.go:310] - The kubelet is not running
I0914 00:57:20.263299 66801 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0914 00:57:20.263306 66801 kubeadm.go:310]
I0914 00:57:20.263397 66801 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0914 00:57:20.263448 66801 kubeadm.go:310] - 'systemctl status kubelet'
I0914 00:57:20.263494 66801 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0914 00:57:20.263516 66801 kubeadm.go:310]
I0914 00:57:20.263632 66801 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0914 00:57:20.263731 66801 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0914 00:57:20.263749 66801 kubeadm.go:310]
I0914 00:57:20.263892 66801 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0914 00:57:20.263985 66801 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0914 00:57:20.264084 66801 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0914 00:57:20.264179 66801 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0914 00:57:20.264188 66801 kubeadm.go:310]
I0914 00:57:20.265123 66801 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0914 00:57:20.265205 66801 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0914 00:57:20.265265 66801 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0914 00:57:20.265332 66801 kubeadm.go:394] duration metric: took 3m55.085697162s to StartCluster
I0914 00:57:20.265368 66801 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0914 00:57:20.265415 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0914 00:57:20.306070 66801 cri.go:89] found id: ""
I0914 00:57:20.306105 66801 logs.go:276] 0 containers: []
W0914 00:57:20.306113 66801 logs.go:278] No container was found matching "kube-apiserver"
I0914 00:57:20.306119 66801 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0914 00:57:20.306175 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0914 00:57:20.338931 66801 cri.go:89] found id: ""
I0914 00:57:20.338962 66801 logs.go:276] 0 containers: []
W0914 00:57:20.338973 66801 logs.go:278] No container was found matching "etcd"
I0914 00:57:20.338982 66801 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0914 00:57:20.339036 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0914 00:57:20.372786 66801 cri.go:89] found id: ""
I0914 00:57:20.372812 66801 logs.go:276] 0 containers: []
W0914 00:57:20.372820 66801 logs.go:278] No container was found matching "coredns"
I0914 00:57:20.372826 66801 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0914 00:57:20.372885 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0914 00:57:20.404358 66801 cri.go:89] found id: ""
I0914 00:57:20.404381 66801 logs.go:276] 0 containers: []
W0914 00:57:20.404390 66801 logs.go:278] No container was found matching "kube-scheduler"
I0914 00:57:20.404397 66801 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0914 00:57:20.404453 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0914 00:57:20.436926 66801 cri.go:89] found id: ""
I0914 00:57:20.436953 66801 logs.go:276] 0 containers: []
W0914 00:57:20.436961 66801 logs.go:278] No container was found matching "kube-proxy"
I0914 00:57:20.436966 66801 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0914 00:57:20.437014 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0914 00:57:20.469703 66801 cri.go:89] found id: ""
I0914 00:57:20.469729 66801 logs.go:276] 0 containers: []
W0914 00:57:20.469737 66801 logs.go:278] No container was found matching "kube-controller-manager"
I0914 00:57:20.469742 66801 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0914 00:57:20.469807 66801 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0914 00:57:20.505821 66801 cri.go:89] found id: ""
I0914 00:57:20.505853 66801 logs.go:276] 0 containers: []
W0914 00:57:20.505861 66801 logs.go:278] No container was found matching "kindnet"
I0914 00:57:20.505870 66801 logs.go:123] Gathering logs for kubelet ...
I0914 00:57:20.505888 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0914 00:57:20.556330 66801 logs.go:123] Gathering logs for dmesg ...
I0914 00:57:20.556366 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0914 00:57:20.569637 66801 logs.go:123] Gathering logs for describe nodes ...
I0914 00:57:20.569661 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0914 00:57:20.678176 66801 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0914 00:57:20.678200 66801 logs.go:123] Gathering logs for CRI-O ...
I0914 00:57:20.678212 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0914 00:57:20.790946 66801 logs.go:123] Gathering logs for container status ...
I0914 00:57:20.790983 66801 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0914 00:57:20.826938 66801 out.go:418] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0914 00:57:20.827011 66801 out.go:270] *
*
W0914 00:57:20.827081 66801 out.go:270] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0914 00:57:20.827101 66801 out.go:270] *
*
W0914 00:57:20.828011 66801 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0914 00:57:20.830648 66801 out.go:201]
W0914 00:57:20.831659 66801 out.go:270] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0914 00:57:20.831701 66801 out.go:270] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0914 00:57:20.831730 66801 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0914 00:57:20.833050 66801 out.go:201]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-431084 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-431084 -n old-k8s-version-431084
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-431084 -n old-k8s-version-431084: exit status 6 (218.81326ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0914 00:57:21.094632 73145 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-431084" does not appear in /home/jenkins/minikube-integration/19640-5422/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-431084" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (285.44s)