=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-366657 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-366657 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m46.94692258s)
-- stdout --
* [old-k8s-version-366657] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19312
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19312-5094/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19312-5094/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-366657" primary control-plane node in "old-k8s-version-366657" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0722 00:40:14.864197 63788 out.go:291] Setting OutFile to fd 1 ...
I0722 00:40:14.864513 63788 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0722 00:40:14.864525 63788 out.go:304] Setting ErrFile to fd 2...
I0722 00:40:14.864531 63788 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0722 00:40:14.864725 63788 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19312-5094/.minikube/bin
I0722 00:40:14.865293 63788 out.go:298] Setting JSON to false
I0722 00:40:14.866376 63788 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":4959,"bootTime":1721603856,"procs":343,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0722 00:40:14.866436 63788 start.go:139] virtualization: kvm guest
I0722 00:40:14.868653 63788 out.go:177] * [old-k8s-version-366657] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0722 00:40:14.870295 63788 out.go:177] - MINIKUBE_LOCATION=19312
I0722 00:40:14.870292 63788 notify.go:220] Checking for updates...
I0722 00:40:14.872779 63788 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0722 00:40:14.873887 63788 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19312-5094/kubeconfig
I0722 00:40:14.875037 63788 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19312-5094/.minikube
I0722 00:40:14.876222 63788 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0722 00:40:14.877484 63788 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0722 00:40:14.879238 63788 config.go:182] Loaded profile config "bridge-280040": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0722 00:40:14.879333 63788 config.go:182] Loaded profile config "enable-default-cni-280040": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0722 00:40:14.879409 63788 config.go:182] Loaded profile config "flannel-280040": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0722 00:40:14.879487 63788 driver.go:392] Setting default libvirt URI to qemu:///system
I0722 00:40:14.921163 63788 out.go:177] * Using the kvm2 driver based on user configuration
I0722 00:40:14.922456 63788 start.go:297] selected driver: kvm2
I0722 00:40:14.922474 63788 start.go:901] validating driver "kvm2" against <nil>
I0722 00:40:14.922488 63788 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0722 00:40:14.923481 63788 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0722 00:40:14.923579 63788 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19312-5094/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0722 00:40:14.938987 63788 install.go:137] /home/jenkins/minikube-integration/19312-5094/.minikube/bin/docker-machine-driver-kvm2 version is 1.33.1
I0722 00:40:14.939032 63788 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0722 00:40:14.939398 63788 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0722 00:40:14.939434 63788 cni.go:84] Creating CNI manager for ""
I0722 00:40:14.939444 63788 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0722 00:40:14.939453 63788 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0722 00:40:14.939561 63788 start.go:340] cluster config:
{Name:old-k8s-version-366657 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721324606-19298@sha256:1c495b056df42bd3fd9a5c30d049e1802f9ed73a342611781f1ccc3c3853953f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-366657 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0722 00:40:14.939712 63788 iso.go:125] acquiring lock: {Name:mk1c358d2514c457d22859dd20040df877cb9d42 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0722 00:40:14.942346 63788 out.go:177] * Starting "old-k8s-version-366657" primary control-plane node in "old-k8s-version-366657" cluster
I0722 00:40:14.943554 63788 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0722 00:40:14.943594 63788 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0722 00:40:14.943611 63788 cache.go:56] Caching tarball of preloaded images
I0722 00:40:14.943685 63788 preload.go:172] Found /home/jenkins/minikube-integration/19312-5094/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0722 00:40:14.943698 63788 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0722 00:40:14.943795 63788 profile.go:143] Saving config to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/config.json ...
I0722 00:40:14.943818 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/config.json: {Name:mk16f7a42e5061d5b8c8cdf0caa84ebda221b2bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:40:14.943986 63788 start.go:360] acquireMachinesLock for old-k8s-version-366657: {Name:mk6b3c50c1c221dd600e48c8652a2f77916f7114 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0722 00:40:36.411036 63788 start.go:364] duration metric: took 21.467012118s to acquireMachinesLock for "old-k8s-version-366657"
I0722 00:40:36.411098 63788 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-366657 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19298/minikube-v1.33.1-1721324531-19298-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721324606-19298@sha256:1c495b056df42bd3fd9a5c30d049e1802f9ed73a342611781f1ccc3c3853953f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-366657 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0722 00:40:36.411205 63788 start.go:125] createHost starting for "" (driver="kvm2")
I0722 00:40:36.413133 63788 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0722 00:40:36.413357 63788 main.go:141] libmachine: Found binary path at /home/jenkins/minikube-integration/19312-5094/.minikube/bin/docker-machine-driver-kvm2
I0722 00:40:36.413403 63788 main.go:141] libmachine: Launching plugin server for driver kvm2
I0722 00:40:36.430790 63788 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:35913
I0722 00:40:36.431218 63788 main.go:141] libmachine: () Calling .GetVersion
I0722 00:40:36.431791 63788 main.go:141] libmachine: Using API Version 1
I0722 00:40:36.431812 63788 main.go:141] libmachine: () Calling .SetConfigRaw
I0722 00:40:36.432192 63788 main.go:141] libmachine: () Calling .GetMachineName
I0722 00:40:36.432402 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetMachineName
I0722 00:40:36.432616 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:36.432910 63788 start.go:159] libmachine.API.Create for "old-k8s-version-366657" (driver="kvm2")
I0722 00:40:36.432964 63788 client.go:168] LocalClient.Create starting
I0722 00:40:36.433012 63788 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem
I0722 00:40:36.433067 63788 main.go:141] libmachine: Decoding PEM data...
I0722 00:40:36.433086 63788 main.go:141] libmachine: Parsing certificate...
I0722 00:40:36.433152 63788 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19312-5094/.minikube/certs/cert.pem
I0722 00:40:36.433182 63788 main.go:141] libmachine: Decoding PEM data...
I0722 00:40:36.433207 63788 main.go:141] libmachine: Parsing certificate...
I0722 00:40:36.433233 63788 main.go:141] libmachine: Running pre-create checks...
I0722 00:40:36.433261 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .PreCreateCheck
I0722 00:40:36.433643 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetConfigRaw
I0722 00:40:36.434082 63788 main.go:141] libmachine: Creating machine...
I0722 00:40:36.434096 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .Create
I0722 00:40:36.434220 63788 main.go:141] libmachine: (old-k8s-version-366657) Creating KVM machine...
I0722 00:40:36.435661 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found existing default KVM network
I0722 00:40:36.436916 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:36.436757 65055 network.go:206] using free private subnet 192.168.39.0/24: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000015720}
I0722 00:40:36.436942 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | created network xml:
I0722 00:40:36.436957 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <network>
I0722 00:40:36.436971 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <name>mk-old-k8s-version-366657</name>
I0722 00:40:36.436980 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <dns enable='no'/>
I0722 00:40:36.436987 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG |
I0722 00:40:36.436997 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <ip address='192.168.39.1' netmask='255.255.255.0'>
I0722 00:40:36.437008 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <dhcp>
I0722 00:40:36.437018 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | <range start='192.168.39.2' end='192.168.39.253'/>
I0722 00:40:36.437031 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | </dhcp>
I0722 00:40:36.437040 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | </ip>
I0722 00:40:36.437052 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG |
I0722 00:40:36.437074 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | </network>
I0722 00:40:36.437083 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG |
I0722 00:40:36.442711 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | trying to create private KVM network mk-old-k8s-version-366657 192.168.39.0/24...
I0722 00:40:36.512910 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | private KVM network mk-old-k8s-version-366657 192.168.39.0/24 created
I0722 00:40:36.512962 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:36.512881 65055 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19312-5094/.minikube
I0722 00:40:36.512987 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting up store path in /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657 ...
I0722 00:40:36.513015 63788 main.go:141] libmachine: (old-k8s-version-366657) Building disk image from file:///home/jenkins/minikube-integration/19312-5094/.minikube/cache/iso/amd64/minikube-v1.33.1-1721324531-19298-amd64.iso
I0722 00:40:36.513034 63788 main.go:141] libmachine: (old-k8s-version-366657) Downloading /home/jenkins/minikube-integration/19312-5094/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19312-5094/.minikube/cache/iso/amd64/minikube-v1.33.1-1721324531-19298-amd64.iso...
I0722 00:40:36.744942 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:36.744803 65055 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa...
I0722 00:40:36.862292 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:36.862149 65055 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/old-k8s-version-366657.rawdisk...
I0722 00:40:36.862326 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Writing magic tar header
I0722 00:40:36.862341 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Writing SSH key tar header
I0722 00:40:36.862354 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:36.862315 65055 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657 ...
I0722 00:40:36.862513 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657
I0722 00:40:36.862548 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657 (perms=drwx------)
I0722 00:40:36.862568 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19312-5094/.minikube/machines
I0722 00:40:36.862583 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins/minikube-integration/19312-5094/.minikube/machines (perms=drwxr-xr-x)
I0722 00:40:36.862595 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19312-5094/.minikube
I0722 00:40:36.862621 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19312-5094
I0722 00:40:36.862636 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins/minikube-integration/19312-5094/.minikube (perms=drwxr-xr-x)
I0722 00:40:36.862649 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0722 00:40:36.862664 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home/jenkins
I0722 00:40:36.862675 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Checking permissions on dir: /home
I0722 00:40:36.862688 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Skipping /home - not owner
I0722 00:40:36.862714 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins/minikube-integration/19312-5094 (perms=drwxrwxr-x)
I0722 00:40:36.862744 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0722 00:40:36.862777 63788 main.go:141] libmachine: (old-k8s-version-366657) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0722 00:40:36.862797 63788 main.go:141] libmachine: (old-k8s-version-366657) Creating domain...
I0722 00:40:36.863768 63788 main.go:141] libmachine: (old-k8s-version-366657) define libvirt domain using xml:
I0722 00:40:36.863784 63788 main.go:141] libmachine: (old-k8s-version-366657) <domain type='kvm'>
I0722 00:40:36.863790 63788 main.go:141] libmachine: (old-k8s-version-366657) <name>old-k8s-version-366657</name>
I0722 00:40:36.863798 63788 main.go:141] libmachine: (old-k8s-version-366657) <memory unit='MiB'>2200</memory>
I0722 00:40:36.863804 63788 main.go:141] libmachine: (old-k8s-version-366657) <vcpu>2</vcpu>
I0722 00:40:36.863811 63788 main.go:141] libmachine: (old-k8s-version-366657) <features>
I0722 00:40:36.863817 63788 main.go:141] libmachine: (old-k8s-version-366657) <acpi/>
I0722 00:40:36.863823 63788 main.go:141] libmachine: (old-k8s-version-366657) <apic/>
I0722 00:40:36.863828 63788 main.go:141] libmachine: (old-k8s-version-366657) <pae/>
I0722 00:40:36.863842 63788 main.go:141] libmachine: (old-k8s-version-366657)
I0722 00:40:36.863850 63788 main.go:141] libmachine: (old-k8s-version-366657) </features>
I0722 00:40:36.863855 63788 main.go:141] libmachine: (old-k8s-version-366657) <cpu mode='host-passthrough'>
I0722 00:40:36.863860 63788 main.go:141] libmachine: (old-k8s-version-366657)
I0722 00:40:36.863864 63788 main.go:141] libmachine: (old-k8s-version-366657) </cpu>
I0722 00:40:36.863883 63788 main.go:141] libmachine: (old-k8s-version-366657) <os>
I0722 00:40:36.863900 63788 main.go:141] libmachine: (old-k8s-version-366657) <type>hvm</type>
I0722 00:40:36.863908 63788 main.go:141] libmachine: (old-k8s-version-366657) <boot dev='cdrom'/>
I0722 00:40:36.863920 63788 main.go:141] libmachine: (old-k8s-version-366657) <boot dev='hd'/>
I0722 00:40:36.863930 63788 main.go:141] libmachine: (old-k8s-version-366657) <bootmenu enable='no'/>
I0722 00:40:36.863942 63788 main.go:141] libmachine: (old-k8s-version-366657) </os>
I0722 00:40:36.863953 63788 main.go:141] libmachine: (old-k8s-version-366657) <devices>
I0722 00:40:36.863966 63788 main.go:141] libmachine: (old-k8s-version-366657) <disk type='file' device='cdrom'>
I0722 00:40:36.863997 63788 main.go:141] libmachine: (old-k8s-version-366657) <source file='/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/boot2docker.iso'/>
I0722 00:40:36.864012 63788 main.go:141] libmachine: (old-k8s-version-366657) <target dev='hdc' bus='scsi'/>
I0722 00:40:36.864023 63788 main.go:141] libmachine: (old-k8s-version-366657) <readonly/>
I0722 00:40:36.864032 63788 main.go:141] libmachine: (old-k8s-version-366657) </disk>
I0722 00:40:36.864046 63788 main.go:141] libmachine: (old-k8s-version-366657) <disk type='file' device='disk'>
I0722 00:40:36.864060 63788 main.go:141] libmachine: (old-k8s-version-366657) <driver name='qemu' type='raw' cache='default' io='threads' />
I0722 00:40:36.864076 63788 main.go:141] libmachine: (old-k8s-version-366657) <source file='/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/old-k8s-version-366657.rawdisk'/>
I0722 00:40:36.864085 63788 main.go:141] libmachine: (old-k8s-version-366657) <target dev='hda' bus='virtio'/>
I0722 00:40:36.864095 63788 main.go:141] libmachine: (old-k8s-version-366657) </disk>
I0722 00:40:36.864108 63788 main.go:141] libmachine: (old-k8s-version-366657) <interface type='network'>
I0722 00:40:36.864124 63788 main.go:141] libmachine: (old-k8s-version-366657) <source network='mk-old-k8s-version-366657'/>
I0722 00:40:36.864136 63788 main.go:141] libmachine: (old-k8s-version-366657) <model type='virtio'/>
I0722 00:40:36.864149 63788 main.go:141] libmachine: (old-k8s-version-366657) </interface>
I0722 00:40:36.864163 63788 main.go:141] libmachine: (old-k8s-version-366657) <interface type='network'>
I0722 00:40:36.864187 63788 main.go:141] libmachine: (old-k8s-version-366657) <source network='default'/>
I0722 00:40:36.864207 63788 main.go:141] libmachine: (old-k8s-version-366657) <model type='virtio'/>
I0722 00:40:36.864217 63788 main.go:141] libmachine: (old-k8s-version-366657) </interface>
I0722 00:40:36.864225 63788 main.go:141] libmachine: (old-k8s-version-366657) <serial type='pty'>
I0722 00:40:36.864238 63788 main.go:141] libmachine: (old-k8s-version-366657) <target port='0'/>
I0722 00:40:36.864249 63788 main.go:141] libmachine: (old-k8s-version-366657) </serial>
I0722 00:40:36.864260 63788 main.go:141] libmachine: (old-k8s-version-366657) <console type='pty'>
I0722 00:40:36.864272 63788 main.go:141] libmachine: (old-k8s-version-366657) <target type='serial' port='0'/>
I0722 00:40:36.864284 63788 main.go:141] libmachine: (old-k8s-version-366657) </console>
I0722 00:40:36.864300 63788 main.go:141] libmachine: (old-k8s-version-366657) <rng model='virtio'>
I0722 00:40:36.864320 63788 main.go:141] libmachine: (old-k8s-version-366657) <backend model='random'>/dev/random</backend>
I0722 00:40:36.864341 63788 main.go:141] libmachine: (old-k8s-version-366657) </rng>
I0722 00:40:36.864355 63788 main.go:141] libmachine: (old-k8s-version-366657)
I0722 00:40:36.864365 63788 main.go:141] libmachine: (old-k8s-version-366657)
I0722 00:40:36.864379 63788 main.go:141] libmachine: (old-k8s-version-366657) </devices>
I0722 00:40:36.864390 63788 main.go:141] libmachine: (old-k8s-version-366657) </domain>
I0722 00:40:36.864406 63788 main.go:141] libmachine: (old-k8s-version-366657)
I0722 00:40:36.868789 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:f3:7a:4d in network default
I0722 00:40:36.869439 63788 main.go:141] libmachine: (old-k8s-version-366657) Ensuring networks are active...
I0722 00:40:36.869469 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:36.870150 63788 main.go:141] libmachine: (old-k8s-version-366657) Ensuring network default is active
I0722 00:40:36.870416 63788 main.go:141] libmachine: (old-k8s-version-366657) Ensuring network mk-old-k8s-version-366657 is active
I0722 00:40:36.870926 63788 main.go:141] libmachine: (old-k8s-version-366657) Getting domain xml...
I0722 00:40:36.871620 63788 main.go:141] libmachine: (old-k8s-version-366657) Creating domain...
I0722 00:40:38.147356 63788 main.go:141] libmachine: (old-k8s-version-366657) Waiting to get IP...
I0722 00:40:38.148343 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:38.148906 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:38.148936 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:38.148876 65055 retry.go:31] will retry after 211.9889ms: waiting for machine to come up
I0722 00:40:38.362549 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:38.363259 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:38.363307 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:38.363229 65055 retry.go:31] will retry after 327.916518ms: waiting for machine to come up
I0722 00:40:38.692940 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:38.693538 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:38.693567 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:38.693504 65055 retry.go:31] will retry after 470.074949ms: waiting for machine to come up
I0722 00:40:39.165829 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:39.166975 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:39.167003 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:39.166925 65055 retry.go:31] will retry after 495.356657ms: waiting for machine to come up
I0722 00:40:39.663591 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:39.664240 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:39.664269 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:39.664196 65055 retry.go:31] will retry after 605.986011ms: waiting for machine to come up
I0722 00:40:40.272357 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:40.272870 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:40.272918 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:40.272821 65055 retry.go:31] will retry after 650.557396ms: waiting for machine to come up
I0722 00:40:40.924766 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:40.925366 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:40.925435 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:40.925309 65055 retry.go:31] will retry after 1.190552524s: waiting for machine to come up
I0722 00:40:42.117701 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:42.118196 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:42.118219 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:42.118159 65055 retry.go:31] will retry after 1.088402701s: waiting for machine to come up
I0722 00:40:43.207780 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:43.208325 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:43.208352 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:43.208262 65055 retry.go:31] will retry after 1.272021705s: waiting for machine to come up
I0722 00:40:44.481511 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:44.481971 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:44.481995 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:44.481940 65055 retry.go:31] will retry after 2.325038205s: waiting for machine to come up
I0722 00:40:46.808374 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:46.808885 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:46.808937 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:46.808845 65055 retry.go:31] will retry after 2.744467052s: waiting for machine to come up
I0722 00:40:49.556838 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:49.557307 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:49.557346 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:49.557273 65055 retry.go:31] will retry after 2.322385755s: waiting for machine to come up
I0722 00:40:51.881882 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:51.882482 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find current IP address of domain old-k8s-version-366657 in network mk-old-k8s-version-366657
I0722 00:40:51.882512 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | I0722 00:40:51.882431 65055 retry.go:31] will retry after 3.850434721s: waiting for machine to come up
I0722 00:40:55.734033 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.734626 63788 main.go:141] libmachine: (old-k8s-version-366657) Found IP for machine: 192.168.39.174
I0722 00:40:55.734652 63788 main.go:141] libmachine: (old-k8s-version-366657) Reserving static IP address...
I0722 00:40:55.734678 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has current primary IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.735018 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-366657", mac: "52:54:00:1a:f7:37", ip: "192.168.39.174"} in network mk-old-k8s-version-366657
I0722 00:40:55.808835 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Getting to WaitForSSH function...
I0722 00:40:55.808861 63788 main.go:141] libmachine: (old-k8s-version-366657) Reserved static IP address: 192.168.39.174
I0722 00:40:55.808877 63788 main.go:141] libmachine: (old-k8s-version-366657) Waiting for SSH to be available...
I0722 00:40:55.811450 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.811933 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:minikube Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:55.811962 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.812214 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Using SSH client type: external
I0722 00:40:55.812241 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Using SSH private key: /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa (-rw-------)
I0722 00:40:55.812287 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.174 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa -p 22] /usr/bin/ssh <nil>}
I0722 00:40:55.812304 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | About to run SSH command:
I0722 00:40:55.812354 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | exit 0
I0722 00:40:55.939490 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | SSH cmd err, output: <nil>:
I0722 00:40:55.939809 63788 main.go:141] libmachine: (old-k8s-version-366657) KVM machine creation complete!
I0722 00:40:55.940105 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetConfigRaw
I0722 00:40:55.940691 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:55.940929 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:55.941094 63788 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0722 00:40:55.941107 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetState
I0722 00:40:55.942477 63788 main.go:141] libmachine: Detecting operating system of created instance...
I0722 00:40:55.942489 63788 main.go:141] libmachine: Waiting for SSH to be available...
I0722 00:40:55.942495 63788 main.go:141] libmachine: Getting to WaitForSSH function...
I0722 00:40:55.942514 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:55.945012 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.945390 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:55.945418 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:55.945547 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:55.945762 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:55.945939 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:55.946088 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:55.946280 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:55.946485 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:55.946498 63788 main.go:141] libmachine: About to run SSH command:
exit 0
I0722 00:40:56.049670 63788 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0722 00:40:56.049695 63788 main.go:141] libmachine: Detecting the provisioner...
I0722 00:40:56.049704 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.052265 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.052619 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.052661 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.052829 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.053011 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.053153 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.053297 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.053448 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:56.053640 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:56.053656 63788 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0722 00:40:56.159146 63788 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0722 00:40:56.159206 63788 main.go:141] libmachine: found compatible host: buildroot
I0722 00:40:56.159212 63788 main.go:141] libmachine: Provisioning with buildroot...
I0722 00:40:56.159219 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetMachineName
I0722 00:40:56.159499 63788 buildroot.go:166] provisioning hostname "old-k8s-version-366657"
I0722 00:40:56.159527 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetMachineName
I0722 00:40:56.159723 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.162313 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.162729 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.162759 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.162887 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.163045 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.163220 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.163361 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.163512 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:56.163692 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:56.163705 63788 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-366657 && echo "old-k8s-version-366657" | sudo tee /etc/hostname
I0722 00:40:56.284165 63788 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-366657
I0722 00:40:56.284200 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.286857 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.287231 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.287260 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.287412 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.287574 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.287694 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.287861 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.288062 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:56.288254 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:56.288282 63788 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-366657' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-366657/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-366657' | sudo tee -a /etc/hosts;
fi
fi
I0722 00:40:56.399377 63788 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0722 00:40:56.399405 63788 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19312-5094/.minikube CaCertPath:/home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19312-5094/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19312-5094/.minikube}
I0722 00:40:56.399445 63788 buildroot.go:174] setting up certificates
I0722 00:40:56.399458 63788 provision.go:84] configureAuth start
I0722 00:40:56.399471 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetMachineName
I0722 00:40:56.399774 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetIP
I0722 00:40:56.402835 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.403259 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.403299 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.403455 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.405653 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.406044 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.406073 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.406339 63788 provision.go:143] copyHostCerts
I0722 00:40:56.406410 63788 exec_runner.go:144] found /home/jenkins/minikube-integration/19312-5094/.minikube/ca.pem, removing ...
I0722 00:40:56.406434 63788 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19312-5094/.minikube/ca.pem
I0722 00:40:56.406508 63788 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19312-5094/.minikube/ca.pem (1082 bytes)
I0722 00:40:56.406635 63788 exec_runner.go:144] found /home/jenkins/minikube-integration/19312-5094/.minikube/cert.pem, removing ...
I0722 00:40:56.406646 63788 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19312-5094/.minikube/cert.pem
I0722 00:40:56.406681 63788 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19312-5094/.minikube/cert.pem (1123 bytes)
I0722 00:40:56.406755 63788 exec_runner.go:144] found /home/jenkins/minikube-integration/19312-5094/.minikube/key.pem, removing ...
I0722 00:40:56.406764 63788 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19312-5094/.minikube/key.pem
I0722 00:40:56.406791 63788 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19312-5094/.minikube/key.pem (1679 bytes)
I0722 00:40:56.406853 63788 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19312-5094/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-366657 san=[127.0.0.1 192.168.39.174 localhost minikube old-k8s-version-366657]
I0722 00:40:56.535086 63788 provision.go:177] copyRemoteCerts
I0722 00:40:56.535140 63788 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0722 00:40:56.535162 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.537860 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.538246 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.538268 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.538477 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.538695 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.538858 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.539000 63788 sshutil.go:53] new ssh client: &{IP:192.168.39.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa Username:docker}
I0722 00:40:56.624852 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0722 00:40:56.649634 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0722 00:40:56.672645 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0722 00:40:56.695355 63788 provision.go:87] duration metric: took 295.884192ms to configureAuth
I0722 00:40:56.695381 63788 buildroot.go:189] setting minikube options for container-runtime
I0722 00:40:56.695564 63788 config.go:182] Loaded profile config "old-k8s-version-366657": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0722 00:40:56.695661 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.698386 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.698828 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.698854 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.699128 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.699342 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.699514 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.699650 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.699898 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:56.700076 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:56.700098 63788 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0722 00:40:56.962853 63788 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0722 00:40:56.962885 63788 main.go:141] libmachine: Checking connection to Docker...
I0722 00:40:56.962896 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetURL
I0722 00:40:56.964074 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | Using libvirt version 6000000
I0722 00:40:56.966386 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.966745 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.966770 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.966946 63788 main.go:141] libmachine: Docker is up and running!
I0722 00:40:56.966962 63788 main.go:141] libmachine: Reticulating splines...
I0722 00:40:56.966970 63788 client.go:171] duration metric: took 20.533992907s to LocalClient.Create
I0722 00:40:56.966998 63788 start.go:167] duration metric: took 20.534089821s to libmachine.API.Create "old-k8s-version-366657"
I0722 00:40:56.967005 63788 start.go:293] postStartSetup for "old-k8s-version-366657" (driver="kvm2")
I0722 00:40:56.967029 63788 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0722 00:40:56.967044 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:56.967249 63788 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0722 00:40:56.967275 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:56.969232 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.969511 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:56.969537 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:56.969693 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:56.969875 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:56.970045 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:56.970165 63788 sshutil.go:53] new ssh client: &{IP:192.168.39.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa Username:docker}
I0722 00:40:57.053628 63788 ssh_runner.go:195] Run: cat /etc/os-release
I0722 00:40:57.057756 63788 info.go:137] Remote host: Buildroot 2023.02.9
I0722 00:40:57.057774 63788 filesync.go:126] Scanning /home/jenkins/minikube-integration/19312-5094/.minikube/addons for local assets ...
I0722 00:40:57.057831 63788 filesync.go:126] Scanning /home/jenkins/minikube-integration/19312-5094/.minikube/files for local assets ...
I0722 00:40:57.057915 63788 filesync.go:149] local asset: /home/jenkins/minikube-integration/19312-5094/.minikube/files/etc/ssl/certs/122632.pem -> 122632.pem in /etc/ssl/certs
I0722 00:40:57.058037 63788 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0722 00:40:57.069402 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/files/etc/ssl/certs/122632.pem --> /etc/ssl/certs/122632.pem (1708 bytes)
I0722 00:40:57.093365 63788 start.go:296] duration metric: took 126.346333ms for postStartSetup
I0722 00:40:57.093418 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetConfigRaw
I0722 00:40:57.093992 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetIP
I0722 00:40:57.096796 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.097261 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:57.097286 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.097538 63788 profile.go:143] Saving config to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/config.json ...
I0722 00:40:57.097772 63788 start.go:128] duration metric: took 20.686554918s to createHost
I0722 00:40:57.097800 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:57.100213 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.100580 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:57.100613 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.100767 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:57.100987 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:57.101170 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:57.101378 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:57.101566 63788 main.go:141] libmachine: Using SSH client type: native
I0722 00:40:57.101763 63788 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.174 22 <nil> <nil>}
I0722 00:40:57.101776 63788 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0722 00:40:57.207150 63788 main.go:141] libmachine: SSH cmd err, output: <nil>: 1721608857.158617630
I0722 00:40:57.207168 63788 fix.go:216] guest clock: 1721608857.158617630
I0722 00:40:57.207175 63788 fix.go:229] Guest: 2024-07-22 00:40:57.15861763 +0000 UTC Remote: 2024-07-22 00:40:57.097786982 +0000 UTC m=+42.277896182 (delta=60.830648ms)
I0722 00:40:57.207193 63788 fix.go:200] guest clock delta is within tolerance: 60.830648ms
I0722 00:40:57.207198 63788 start.go:83] releasing machines lock for "old-k8s-version-366657", held for 20.796135961s
I0722 00:40:57.207235 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:57.207595 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetIP
I0722 00:40:57.210478 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.210960 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:57.211002 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.211122 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:57.211641 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:57.211812 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .DriverName
I0722 00:40:57.211915 63788 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0722 00:40:57.211952 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:57.212013 63788 ssh_runner.go:195] Run: cat /version.json
I0722 00:40:57.212037 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHHostname
I0722 00:40:57.215028 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.215154 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.215536 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:57.215768 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHPort
I0722 00:40:57.215774 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:57.215843 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:57.215891 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.215921 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:57.215937 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:57.215944 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHKeyPath
I0722 00:40:57.215955 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:57.216213 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetSSHUsername
I0722 00:40:57.216248 63788 sshutil.go:53] new ssh client: &{IP:192.168.39.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa Username:docker}
I0722 00:40:57.216342 63788 sshutil.go:53] new ssh client: &{IP:192.168.39.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19312-5094/.minikube/machines/old-k8s-version-366657/id_rsa Username:docker}
I0722 00:40:57.337972 63788 ssh_runner.go:195] Run: systemctl --version
I0722 00:40:57.344804 63788 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0722 00:40:57.523322 63788 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0722 00:40:57.529890 63788 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0722 00:40:57.529984 63788 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0722 00:40:57.545936 63788 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0722 00:40:57.545967 63788 start.go:495] detecting cgroup driver to use...
I0722 00:40:57.546041 63788 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0722 00:40:57.563935 63788 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0722 00:40:57.578313 63788 docker.go:217] disabling cri-docker service (if available) ...
I0722 00:40:57.578386 63788 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0722 00:40:57.593182 63788 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0722 00:40:57.607092 63788 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0722 00:40:57.726412 63788 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0722 00:40:57.908921 63788 docker.go:233] disabling docker service ...
I0722 00:40:57.909011 63788 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0722 00:40:57.934152 63788 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0722 00:40:57.946879 63788 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0722 00:40:58.078791 63788 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0722 00:40:58.201628 63788 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0722 00:40:58.214938 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0722 00:40:58.235222 63788 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0722 00:40:58.235289 63788 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0722 00:40:58.245492 63788 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0722 00:40:58.245583 63788 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0722 00:40:58.256617 63788 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0722 00:40:58.266335 63788 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0722 00:40:58.276027 63788 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0722 00:40:58.286091 63788 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0722 00:40:58.295387 63788 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0722 00:40:58.295435 63788 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0722 00:40:58.307075 63788 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0722 00:40:58.320169 63788 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0722 00:40:58.448242 63788 ssh_runner.go:195] Run: sudo systemctl restart crio
I0722 00:40:58.634182 63788 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0722 00:40:58.634268 63788 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0722 00:40:58.640090 63788 start.go:563] Will wait 60s for crictl version
I0722 00:40:58.640150 63788 ssh_runner.go:195] Run: which crictl
I0722 00:40:58.644897 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0722 00:40:58.692031 63788 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0722 00:40:58.692118 63788 ssh_runner.go:195] Run: crio --version
I0722 00:40:58.722174 63788 ssh_runner.go:195] Run: crio --version
I0722 00:40:58.755311 63788 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0722 00:40:58.756550 63788 main.go:141] libmachine: (old-k8s-version-366657) Calling .GetIP
I0722 00:40:58.760209 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:58.760660 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:1a:f7:37", ip: ""} in network mk-old-k8s-version-366657: {Iface:virbr1 ExpiryTime:2024-07-22 01:40:50 +0000 UTC Type:0 Mac:52:54:00:1a:f7:37 Iaid: IPaddr:192.168.39.174 Prefix:24 Hostname:old-k8s-version-366657 Clientid:01:52:54:00:1a:f7:37}
I0722 00:40:58.760696 63788 main.go:141] libmachine: (old-k8s-version-366657) DBG | domain old-k8s-version-366657 has defined IP address 192.168.39.174 and MAC address 52:54:00:1a:f7:37 in network mk-old-k8s-version-366657
I0722 00:40:58.760924 63788 ssh_runner.go:195] Run: grep 192.168.39.1 host.minikube.internal$ /etc/hosts
I0722 00:40:58.765192 63788 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.39.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0722 00:40:58.777306 63788 kubeadm.go:883] updating cluster {Name:old-k8s-version-366657 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19298/minikube-v1.33.1-1721324531-19298-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721324606-19298@sha256:1c495b056df42bd3fd9a5c30d049e1802f9ed73a342611781f1ccc3c3853953f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-366657 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.174 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] M
ountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0722 00:40:58.777460 63788 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0722 00:40:58.777524 63788 ssh_runner.go:195] Run: sudo crictl images --output json
I0722 00:40:58.808074 63788 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0722 00:40:58.808149 63788 ssh_runner.go:195] Run: which lz4
I0722 00:40:58.811978 63788 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0722 00:40:58.816155 63788 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0722 00:40:58.816190 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0722 00:41:00.471709 63788 crio.go:462] duration metric: took 1.659758315s to copy over tarball
I0722 00:41:00.471845 63788 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0722 00:41:03.244166 63788 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.772286838s)
I0722 00:41:03.244194 63788 crio.go:469] duration metric: took 2.772428037s to extract the tarball
I0722 00:41:03.244202 63788 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0722 00:41:03.289838 63788 ssh_runner.go:195] Run: sudo crictl images --output json
I0722 00:41:03.336689 63788 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0722 00:41:03.336710 63788 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0722 00:41:03.336783 63788 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0722 00:41:03.336803 63788 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0722 00:41:03.336812 63788 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0722 00:41:03.336787 63788 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0722 00:41:03.336838 63788 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0722 00:41:03.336858 63788 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0722 00:41:03.336813 63788 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0722 00:41:03.336787 63788 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0722 00:41:03.338270 63788 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0722 00:41:03.338359 63788 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0722 00:41:03.338489 63788 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0722 00:41:03.338532 63788 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0722 00:41:03.338536 63788 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0722 00:41:03.338274 63788 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0722 00:41:03.338698 63788 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0722 00:41:03.338835 63788 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0722 00:41:03.571117 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0722 00:41:03.582350 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0722 00:41:03.594334 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0722 00:41:03.603346 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0722 00:41:03.609233 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0722 00:41:03.611784 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0722 00:41:03.639242 63788 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0722 00:41:03.639302 63788 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0722 00:41:03.639362 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.685065 63788 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0722 00:41:03.685118 63788 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0722 00:41:03.685168 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.730724 63788 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0722 00:41:03.730789 63788 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0722 00:41:03.730849 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.738132 63788 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0722 00:41:03.738167 63788 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0722 00:41:03.738177 63788 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0722 00:41:03.738197 63788 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0722 00:41:03.738219 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.738235 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.738236 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0722 00:41:03.738265 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0722 00:41:03.738136 63788 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0722 00:41:03.738203 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0722 00:41:03.738284 63788 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0722 00:41:03.738306 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.756180 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0722 00:41:03.840864 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0722 00:41:03.840917 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0722 00:41:03.840971 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0722 00:41:03.840982 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0722 00:41:03.841022 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0722 00:41:03.841060 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0722 00:41:03.850157 63788 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0722 00:41:03.850200 63788 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0722 00:41:03.850249 63788 ssh_runner.go:195] Run: which crictl
I0722 00:41:03.928895 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0722 00:41:03.928972 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0722 00:41:03.929024 63788 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0722 00:41:03.929093 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0722 00:41:03.962403 63788 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0722 00:41:04.219751 63788 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0722 00:41:04.372364 63788 cache_images.go:92] duration metric: took 1.035636633s to LoadCachedImages
W0722 00:41:04.372457 63788 out.go:239] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19312-5094/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0722 00:41:04.372473 63788 kubeadm.go:934] updating node { 192.168.39.174 8443 v1.20.0 crio true true} ...
I0722 00:41:04.372623 63788 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-366657 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.39.174
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-366657 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0722 00:41:04.372702 63788 ssh_runner.go:195] Run: crio config
I0722 00:41:04.433786 63788 cni.go:84] Creating CNI manager for ""
I0722 00:41:04.433813 63788 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0722 00:41:04.433828 63788 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0722 00:41:04.433853 63788 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.174 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-366657 NodeName:old-k8s-version-366657 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.174"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.39.174 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt St
aticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0722 00:41:04.434023 63788 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.39.174
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-366657"
kubeletExtraArgs:
node-ip: 192.168.39.174
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.39.174"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0722 00:41:04.434095 63788 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0722 00:41:04.444398 63788 binaries.go:44] Found k8s binaries, skipping transfer
I0722 00:41:04.444486 63788 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0722 00:41:04.454923 63788 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (430 bytes)
I0722 00:41:04.471443 63788 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0722 00:41:04.490245 63788 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2123 bytes)
I0722 00:41:04.510154 63788 ssh_runner.go:195] Run: grep 192.168.39.174 control-plane.minikube.internal$ /etc/hosts
I0722 00:41:04.514505 63788 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.39.174 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0722 00:41:04.531169 63788 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0722 00:41:04.655115 63788 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0722 00:41:04.671926 63788 certs.go:68] Setting up /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657 for IP: 192.168.39.174
I0722 00:41:04.671953 63788 certs.go:194] generating shared ca certs ...
I0722 00:41:04.671972 63788 certs.go:226] acquiring lock for ca certs: {Name:mk670e7dec7f1b116dfecf047bc459d9ed15ed73 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:04.672195 63788 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19312-5094/.minikube/ca.key
I0722 00:41:04.672253 63788 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19312-5094/.minikube/proxy-client-ca.key
I0722 00:41:04.672267 63788 certs.go:256] generating profile certs ...
I0722 00:41:04.672340 63788 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.key
I0722 00:41:04.672370 63788 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.crt with IP's: []
I0722 00:41:04.722033 63788 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.crt ...
I0722 00:41:04.722059 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.crt: {Name:mkeef7389eea3e98c1af995c4d622042ada7b12f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:04.722254 63788 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.key ...
I0722 00:41:04.722277 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/client.key: {Name:mk4ae1f47f8c8751b2d68658bade7fa8c0b9f435 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:04.722420 63788 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key.2cc8579c
I0722 00:41:04.722441 63788 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt.2cc8579c with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.39.174]
I0722 00:41:04.888376 63788 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt.2cc8579c ...
I0722 00:41:04.888401 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt.2cc8579c: {Name:mkf0bd8121b24cc10e856ce50301bc7f1fae66b4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:04.888555 63788 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key.2cc8579c ...
I0722 00:41:04.888571 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key.2cc8579c: {Name:mke97985dd84f9e50a214b2d218c2fbf4bb4e447 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:04.888665 63788 certs.go:381] copying /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt.2cc8579c -> /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt
I0722 00:41:04.888765 63788 certs.go:385] copying /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key.2cc8579c -> /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key
I0722 00:41:04.888839 63788 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.key
I0722 00:41:04.888859 63788 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.crt with IP's: []
I0722 00:41:05.045793 63788 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.crt ...
I0722 00:41:05.045823 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.crt: {Name:mkb6b9b0c7d440aa01d75af3726386d547d278b8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:05.045977 63788 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.key ...
I0722 00:41:05.045990 63788 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.key: {Name:mk3cef6e78f2d7d8d5b5bf913dd7e963af49ad5b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0722 00:41:05.046146 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/12263.pem (1338 bytes)
W0722 00:41:05.046184 63788 certs.go:480] ignoring /home/jenkins/minikube-integration/19312-5094/.minikube/certs/12263_empty.pem, impossibly tiny 0 bytes
I0722 00:41:05.046193 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca-key.pem (1679 bytes)
I0722 00:41:05.046212 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/ca.pem (1082 bytes)
I0722 00:41:05.046235 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/cert.pem (1123 bytes)
I0722 00:41:05.046255 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/certs/key.pem (1679 bytes)
I0722 00:41:05.046292 63788 certs.go:484] found cert: /home/jenkins/minikube-integration/19312-5094/.minikube/files/etc/ssl/certs/122632.pem (1708 bytes)
I0722 00:41:05.046889 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0722 00:41:05.075256 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0722 00:41:05.102201 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0722 00:41:05.125895 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0722 00:41:05.150555 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0722 00:41:05.174535 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0722 00:41:05.198664 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0722 00:41:05.227518 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/profiles/old-k8s-version-366657/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0722 00:41:05.349249 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/files/etc/ssl/certs/122632.pem --> /usr/share/ca-certificates/122632.pem (1708 bytes)
I0722 00:41:05.371894 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0722 00:41:05.397553 63788 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19312-5094/.minikube/certs/12263.pem --> /usr/share/ca-certificates/12263.pem (1338 bytes)
I0722 00:41:05.424496 63788 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0722 00:41:05.442516 63788 ssh_runner.go:195] Run: openssl version
I0722 00:41:05.448591 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/122632.pem && ln -fs /usr/share/ca-certificates/122632.pem /etc/ssl/certs/122632.pem"
I0722 00:41:05.463466 63788 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/122632.pem
I0722 00:41:05.468999 63788 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jul 21 23:37 /usr/share/ca-certificates/122632.pem
I0722 00:41:05.469088 63788 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/122632.pem
I0722 00:41:05.476663 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/122632.pem /etc/ssl/certs/3ec20f2e.0"
I0722 00:41:05.491161 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0722 00:41:05.506192 63788 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0722 00:41:05.512363 63788 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jul 21 23:26 /usr/share/ca-certificates/minikubeCA.pem
I0722 00:41:05.512431 63788 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0722 00:41:05.522079 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0722 00:41:05.540490 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/12263.pem && ln -fs /usr/share/ca-certificates/12263.pem /etc/ssl/certs/12263.pem"
I0722 00:41:05.559377 63788 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/12263.pem
I0722 00:41:05.564423 63788 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jul 21 23:37 /usr/share/ca-certificates/12263.pem
I0722 00:41:05.564491 63788 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/12263.pem
I0722 00:41:05.570191 63788 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/12263.pem /etc/ssl/certs/51391683.0"
I0722 00:41:05.584886 63788 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0722 00:41:05.589594 63788 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0722 00:41:05.589671 63788 kubeadm.go:392] StartCluster: {Name:old-k8s-version-366657 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19298/minikube-v1.33.1-1721324531-19298-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721324606-19298@sha256:1c495b056df42bd3fd9a5c30d049e1802f9ed73a342611781f1ccc3c3853953f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-366657 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.174 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Moun
tPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0722 00:41:05.589780 63788 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0722 00:41:05.589853 63788 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0722 00:41:05.632391 63788 cri.go:89] found id: ""
I0722 00:41:05.632468 63788 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0722 00:41:05.643622 63788 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0722 00:41:05.653867 63788 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0722 00:41:05.664705 63788 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0722 00:41:05.664724 63788 kubeadm.go:157] found existing configuration files:
I0722 00:41:05.664774 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0722 00:41:05.674309 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0722 00:41:05.674374 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0722 00:41:05.684533 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0722 00:41:05.693822 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0722 00:41:05.693878 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0722 00:41:05.705122 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0722 00:41:05.714727 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0722 00:41:05.714795 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0722 00:41:05.724507 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0722 00:41:05.734180 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0722 00:41:05.734242 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0722 00:41:05.746432 63788 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0722 00:41:06.030230 63788 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0722 00:43:04.330703 63788 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0722 00:43:04.330835 63788 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0722 00:43:04.331997 63788 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0722 00:43:04.332066 63788 kubeadm.go:310] [preflight] Running pre-flight checks
I0722 00:43:04.332158 63788 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0722 00:43:04.332296 63788 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0722 00:43:04.332447 63788 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0722 00:43:04.332533 63788 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0722 00:43:04.334035 63788 out.go:204] - Generating certificates and keys ...
I0722 00:43:04.334118 63788 kubeadm.go:310] [certs] Using existing ca certificate authority
I0722 00:43:04.334194 63788 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0722 00:43:04.334273 63788 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0722 00:43:04.334353 63788 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0722 00:43:04.334446 63788 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0722 00:43:04.334523 63788 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0722 00:43:04.334599 63788 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0722 00:43:04.334793 63788 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
I0722 00:43:04.334875 63788 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0722 00:43:04.335036 63788 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
I0722 00:43:04.335137 63788 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0722 00:43:04.335218 63788 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0722 00:43:04.335288 63788 kubeadm.go:310] [certs] Generating "sa" key and public key
I0722 00:43:04.335372 63788 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0722 00:43:04.335445 63788 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0722 00:43:04.335519 63788 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0722 00:43:04.335605 63788 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0722 00:43:04.335682 63788 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0722 00:43:04.335817 63788 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0722 00:43:04.335919 63788 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0722 00:43:04.335973 63788 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0722 00:43:04.336061 63788 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0722 00:43:04.338189 63788 out.go:204] - Booting up control plane ...
I0722 00:43:04.338276 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0722 00:43:04.338360 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0722 00:43:04.338459 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0722 00:43:04.338557 63788 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0722 00:43:04.338742 63788 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0722 00:43:04.338808 63788 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0722 00:43:04.338911 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:04.339198 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:04.339262 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:04.339506 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:04.339602 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:04.339853 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:04.339914 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:04.340069 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:04.340146 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:04.340330 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:04.340341 63788 kubeadm.go:310]
I0722 00:43:04.340404 63788 kubeadm.go:310] Unfortunately, an error has occurred:
I0722 00:43:04.340464 63788 kubeadm.go:310] timed out waiting for the condition
I0722 00:43:04.340473 63788 kubeadm.go:310]
I0722 00:43:04.340522 63788 kubeadm.go:310] This error is likely caused by:
I0722 00:43:04.340570 63788 kubeadm.go:310] - The kubelet is not running
I0722 00:43:04.340704 63788 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0722 00:43:04.340714 63788 kubeadm.go:310]
I0722 00:43:04.340814 63788 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0722 00:43:04.340864 63788 kubeadm.go:310] - 'systemctl status kubelet'
I0722 00:43:04.340893 63788 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0722 00:43:04.340899 63788 kubeadm.go:310]
I0722 00:43:04.341023 63788 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0722 00:43:04.341123 63788 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0722 00:43:04.341132 63788 kubeadm.go:310]
I0722 00:43:04.341247 63788 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0722 00:43:04.341357 63788 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0722 00:43:04.341449 63788 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0722 00:43:04.341527 63788 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0722 00:43:04.341550 63788 kubeadm.go:310]
W0722 00:43:04.341657 63788 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-366657] and IPs [192.168.39.174 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0722 00:43:04.341715 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0722 00:43:04.877939 63788 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0722 00:43:04.891339 63788 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0722 00:43:04.900158 63788 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0722 00:43:04.900178 63788 kubeadm.go:157] found existing configuration files:
I0722 00:43:04.900220 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0722 00:43:04.908509 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0722 00:43:04.908562 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0722 00:43:04.917327 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0722 00:43:04.925565 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0722 00:43:04.925624 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0722 00:43:04.934366 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0722 00:43:04.942515 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0722 00:43:04.942562 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0722 00:43:04.951628 63788 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0722 00:43:04.959768 63788 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0722 00:43:04.959831 63788 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0722 00:43:04.968163 63788 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0722 00:43:05.042731 63788 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0722 00:43:05.042862 63788 kubeadm.go:310] [preflight] Running pre-flight checks
I0722 00:43:05.175659 63788 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0722 00:43:05.175777 63788 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0722 00:43:05.175874 63788 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0722 00:43:05.350147 63788 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0722 00:43:05.352066 63788 out.go:204] - Generating certificates and keys ...
I0722 00:43:05.352152 63788 kubeadm.go:310] [certs] Using existing ca certificate authority
I0722 00:43:05.352238 63788 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0722 00:43:05.352344 63788 kubeadm.go:310] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0722 00:43:05.352433 63788 kubeadm.go:310] [certs] Using existing front-proxy-ca certificate authority
I0722 00:43:05.352533 63788 kubeadm.go:310] [certs] Using existing front-proxy-client certificate and key on disk
I0722 00:43:05.352611 63788 kubeadm.go:310] [certs] Using existing etcd/ca certificate authority
I0722 00:43:05.352723 63788 kubeadm.go:310] [certs] Using existing etcd/server certificate and key on disk
I0722 00:43:05.352813 63788 kubeadm.go:310] [certs] Using existing etcd/peer certificate and key on disk
I0722 00:43:05.352906 63788 kubeadm.go:310] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0722 00:43:05.352974 63788 kubeadm.go:310] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0722 00:43:05.353006 63788 kubeadm.go:310] [certs] Using the existing "sa" key
I0722 00:43:05.353103 63788 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0722 00:43:05.615697 63788 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0722 00:43:05.812975 63788 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0722 00:43:05.922865 63788 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0722 00:43:06.057114 63788 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0722 00:43:06.071636 63788 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0722 00:43:06.072640 63788 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0722 00:43:06.072683 63788 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0722 00:43:06.203837 63788 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0722 00:43:06.205624 63788 out.go:204] - Booting up control plane ...
I0722 00:43:06.205749 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0722 00:43:06.213523 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0722 00:43:06.214463 63788 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0722 00:43:06.215214 63788 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0722 00:43:06.217265 63788 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0722 00:43:46.217581 63788 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0722 00:43:46.218000 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:46.218280 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:43:51.218483 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:43:51.218697 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:44:01.219272 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:44:01.219499 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:44:21.220472 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:44:21.220644 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:45:01.223191 63788 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0722 00:45:01.223695 63788 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0722 00:45:01.223733 63788 kubeadm.go:310]
I0722 00:45:01.223830 63788 kubeadm.go:310] Unfortunately, an error has occurred:
I0722 00:45:01.223924 63788 kubeadm.go:310] timed out waiting for the condition
I0722 00:45:01.223933 63788 kubeadm.go:310]
I0722 00:45:01.224012 63788 kubeadm.go:310] This error is likely caused by:
I0722 00:45:01.224114 63788 kubeadm.go:310] - The kubelet is not running
I0722 00:45:01.224351 63788 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0722 00:45:01.224364 63788 kubeadm.go:310]
I0722 00:45:01.224592 63788 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0722 00:45:01.224694 63788 kubeadm.go:310] - 'systemctl status kubelet'
I0722 00:45:01.224774 63788 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0722 00:45:01.224788 63788 kubeadm.go:310]
I0722 00:45:01.225049 63788 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0722 00:45:01.225240 63788 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0722 00:45:01.225254 63788 kubeadm.go:310]
I0722 00:45:01.225653 63788 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0722 00:45:01.225879 63788 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0722 00:45:01.226125 63788 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0722 00:45:01.226511 63788 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0722 00:45:01.226526 63788 kubeadm.go:310]
I0722 00:45:01.226658 63788 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0722 00:45:01.226746 63788 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0722 00:45:01.226829 63788 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0722 00:45:01.226902 63788 kubeadm.go:394] duration metric: took 3m55.637231397s to StartCluster
I0722 00:45:01.226962 63788 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0722 00:45:01.227061 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0722 00:45:01.266801 63788 cri.go:89] found id: ""
I0722 00:45:01.266826 63788 logs.go:276] 0 containers: []
W0722 00:45:01.266834 63788 logs.go:278] No container was found matching "kube-apiserver"
I0722 00:45:01.266840 63788 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0722 00:45:01.266901 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0722 00:45:01.297938 63788 cri.go:89] found id: ""
I0722 00:45:01.297966 63788 logs.go:276] 0 containers: []
W0722 00:45:01.297977 63788 logs.go:278] No container was found matching "etcd"
I0722 00:45:01.297985 63788 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0722 00:45:01.298051 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0722 00:45:01.330235 63788 cri.go:89] found id: ""
I0722 00:45:01.330259 63788 logs.go:276] 0 containers: []
W0722 00:45:01.330267 63788 logs.go:278] No container was found matching "coredns"
I0722 00:45:01.330273 63788 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0722 00:45:01.330323 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0722 00:45:01.361716 63788 cri.go:89] found id: ""
I0722 00:45:01.361742 63788 logs.go:276] 0 containers: []
W0722 00:45:01.361752 63788 logs.go:278] No container was found matching "kube-scheduler"
I0722 00:45:01.361760 63788 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0722 00:45:01.361826 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0722 00:45:01.392531 63788 cri.go:89] found id: ""
I0722 00:45:01.392556 63788 logs.go:276] 0 containers: []
W0722 00:45:01.392562 63788 logs.go:278] No container was found matching "kube-proxy"
I0722 00:45:01.392568 63788 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0722 00:45:01.392620 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0722 00:45:01.424005 63788 cri.go:89] found id: ""
I0722 00:45:01.424033 63788 logs.go:276] 0 containers: []
W0722 00:45:01.424043 63788 logs.go:278] No container was found matching "kube-controller-manager"
I0722 00:45:01.424051 63788 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0722 00:45:01.424117 63788 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0722 00:45:01.455084 63788 cri.go:89] found id: ""
I0722 00:45:01.455116 63788 logs.go:276] 0 containers: []
W0722 00:45:01.455126 63788 logs.go:278] No container was found matching "kindnet"
I0722 00:45:01.455137 63788 logs.go:123] Gathering logs for container status ...
I0722 00:45:01.455161 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0722 00:45:01.490701 63788 logs.go:123] Gathering logs for kubelet ...
I0722 00:45:01.490733 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0722 00:45:01.537534 63788 logs.go:123] Gathering logs for dmesg ...
I0722 00:45:01.537571 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0722 00:45:01.549731 63788 logs.go:123] Gathering logs for describe nodes ...
I0722 00:45:01.549759 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0722 00:45:01.657580 63788 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0722 00:45:01.657599 63788 logs.go:123] Gathering logs for CRI-O ...
I0722 00:45:01.657610 63788 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
W0722 00:45:01.751188 63788 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0722 00:45:01.751236 63788 out.go:239] *
*
W0722 00:45:01.751298 63788 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0722 00:45:01.751335 63788 out.go:239] *
*
W0722 00:45:01.752278 63788 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0722 00:45:01.755018 63788 out.go:177]
W0722 00:45:01.756138 63788 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0722 00:45:01.756188 63788 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0722 00:45:01.756214 63788 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0722 00:45:01.757551 63788 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-366657 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-366657 -n old-k8s-version-366657
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-366657 -n old-k8s-version-366657: exit status 6 (220.110202ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0722 00:45:02.021983 71017 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-366657" does not appear in /home/jenkins/minikube-integration/19312-5094/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-366657" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (287.22s)