=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-914355 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-914355 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m55.050407329s)
-- stdout --
* [old-k8s-version-914355] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19195
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19195-5988/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19195-5988/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-914355" primary control-plane node in "old-k8s-version-914355" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0708 20:41:59.649790 49548 out.go:291] Setting OutFile to fd 1 ...
I0708 20:41:59.649887 49548 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0708 20:41:59.649892 49548 out.go:304] Setting ErrFile to fd 2...
I0708 20:41:59.649898 49548 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0708 20:41:59.650147 49548 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19195-5988/.minikube/bin
I0708 20:41:59.650951 49548 out.go:298] Setting JSON to false
I0708 20:41:59.651885 49548 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-13","uptime":5069,"bootTime":1720466251,"procs":182,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0708 20:41:59.651946 49548 start.go:139] virtualization: kvm guest
I0708 20:41:59.653772 49548 out.go:177] * [old-k8s-version-914355] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0708 20:41:59.655987 49548 notify.go:220] Checking for updates...
I0708 20:41:59.657363 49548 out.go:177] - MINIKUBE_LOCATION=19195
I0708 20:41:59.658771 49548 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0708 20:41:59.662283 49548 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19195-5988/kubeconfig
I0708 20:41:59.664569 49548 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19195-5988/.minikube
I0708 20:41:59.667086 49548 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0708 20:41:59.670019 49548 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0708 20:41:59.671521 49548 driver.go:392] Setting default libvirt URI to qemu:///system
I0708 20:41:59.710109 49548 out.go:177] * Using the kvm2 driver based on user configuration
I0708 20:41:59.711812 49548 start.go:297] selected driver: kvm2
I0708 20:41:59.711934 49548 start.go:901] validating driver "kvm2" against <nil>
I0708 20:41:59.711959 49548 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0708 20:41:59.713047 49548 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0708 20:41:59.735938 49548 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19195-5988/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0708 20:41:59.753486 49548 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0708 20:41:59.753529 49548 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0708 20:41:59.753764 49548 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0708 20:41:59.753829 49548 cni.go:84] Creating CNI manager for ""
I0708 20:41:59.753844 49548 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0708 20:41:59.753858 49548 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0708 20:41:59.753923 49548 start.go:340] cluster config:
{Name:old-k8s-version-914355 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1720012048-19186@sha256:0fc826bca29cbb5a8335aaf40b2c19a34f0a6b85133ca47842ce6e575d3bc2ef Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-914355 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0708 20:41:59.754054 49548 iso.go:125] acquiring lock: {Name:mkb5cc5061ba7accede97e12b0ec4ee3df03bec6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0708 20:41:59.755744 49548 out.go:177] * Starting "old-k8s-version-914355" primary control-plane node in "old-k8s-version-914355" cluster
I0708 20:41:59.756951 49548 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0708 20:41:59.756993 49548 preload.go:147] Found local preload: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0708 20:41:59.757005 49548 cache.go:56] Caching tarball of preloaded images
I0708 20:41:59.757112 49548 preload.go:173] Found /home/jenkins/minikube-integration/19195-5988/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0708 20:41:59.757127 49548 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0708 20:41:59.757573 49548 profile.go:143] Saving config to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/config.json ...
I0708 20:41:59.757604 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/config.json: {Name:mk2a3161242d6c0771cce516f169d56cc007d51b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:41:59.757795 49548 start.go:360] acquireMachinesLock for old-k8s-version-914355: {Name:mk02da8a2ea9557af4e83be5d1f75b14760e19ea Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0708 20:42:25.424618 49548 start.go:364] duration metric: took 25.666793473s to acquireMachinesLock for "old-k8s-version-914355"
I0708 20:42:25.424694 49548 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-914355 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19186/minikube-v1.33.1-1720011972-19186-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1720012048-19186@sha256:0fc826bca29cbb5a8335aaf40b2c19a34f0a6b85133ca47842ce6e575d3bc2ef Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-914355 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0708 20:42:25.424802 49548 start.go:125] createHost starting for "" (driver="kvm2")
I0708 20:42:25.427089 49548 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0708 20:42:25.427374 49548 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0708 20:42:25.427438 49548 main.go:141] libmachine: Launching plugin server for driver kvm2
I0708 20:42:25.445124 49548 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:44683
I0708 20:42:25.445568 49548 main.go:141] libmachine: () Calling .GetVersion
I0708 20:42:25.446160 49548 main.go:141] libmachine: Using API Version 1
I0708 20:42:25.446181 49548 main.go:141] libmachine: () Calling .SetConfigRaw
I0708 20:42:25.446546 49548 main.go:141] libmachine: () Calling .GetMachineName
I0708 20:42:25.446748 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetMachineName
I0708 20:42:25.446895 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:25.447050 49548 start.go:159] libmachine.API.Create for "old-k8s-version-914355" (driver="kvm2")
I0708 20:42:25.447076 49548 client.go:168] LocalClient.Create starting
I0708 20:42:25.447103 49548 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem
I0708 20:42:25.447141 49548 main.go:141] libmachine: Decoding PEM data...
I0708 20:42:25.447159 49548 main.go:141] libmachine: Parsing certificate...
I0708 20:42:25.447209 49548 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19195-5988/.minikube/certs/cert.pem
I0708 20:42:25.447226 49548 main.go:141] libmachine: Decoding PEM data...
I0708 20:42:25.447238 49548 main.go:141] libmachine: Parsing certificate...
I0708 20:42:25.447253 49548 main.go:141] libmachine: Running pre-create checks...
I0708 20:42:25.447264 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .PreCreateCheck
I0708 20:42:25.447591 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetConfigRaw
I0708 20:42:25.447964 49548 main.go:141] libmachine: Creating machine...
I0708 20:42:25.447977 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .Create
I0708 20:42:25.448101 49548 main.go:141] libmachine: (old-k8s-version-914355) Creating KVM machine...
I0708 20:42:25.449259 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found existing default KVM network
I0708 20:42:25.450036 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.449888 49893 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:74:97:91} reservation:<nil>}
I0708 20:42:25.450812 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.450733 49893 network.go:206] using free private subnet 192.168.50.0/24: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00011da50}
I0708 20:42:25.450845 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | created network xml:
I0708 20:42:25.450859 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <network>
I0708 20:42:25.450875 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <name>mk-old-k8s-version-914355</name>
I0708 20:42:25.450884 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <dns enable='no'/>
I0708 20:42:25.450895 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG |
I0708 20:42:25.450912 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <ip address='192.168.50.1' netmask='255.255.255.0'>
I0708 20:42:25.450940 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <dhcp>
I0708 20:42:25.450982 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | <range start='192.168.50.2' end='192.168.50.253'/>
I0708 20:42:25.450997 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | </dhcp>
I0708 20:42:25.451008 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | </ip>
I0708 20:42:25.451023 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG |
I0708 20:42:25.451030 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | </network>
I0708 20:42:25.451044 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG |
I0708 20:42:25.455747 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | trying to create private KVM network mk-old-k8s-version-914355 192.168.50.0/24...
I0708 20:42:25.524025 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | private KVM network mk-old-k8s-version-914355 192.168.50.0/24 created
I0708 20:42:25.524175 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting up store path in /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355 ...
I0708 20:42:25.524206 49548 main.go:141] libmachine: (old-k8s-version-914355) Building disk image from file:///home/jenkins/minikube-integration/19195-5988/.minikube/cache/iso/amd64/minikube-v1.33.1-1720011972-19186-amd64.iso
I0708 20:42:25.524219 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.524162 49893 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19195-5988/.minikube
I0708 20:42:25.524296 49548 main.go:141] libmachine: (old-k8s-version-914355) Downloading /home/jenkins/minikube-integration/19195-5988/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19195-5988/.minikube/cache/iso/amd64/minikube-v1.33.1-1720011972-19186-amd64.iso...
I0708 20:42:25.748467 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.748344 49893 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa...
I0708 20:42:25.849258 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.849138 49893 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/old-k8s-version-914355.rawdisk...
I0708 20:42:25.849285 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Writing magic tar header
I0708 20:42:25.849341 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Writing SSH key tar header
I0708 20:42:25.849395 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:25.849251 49893 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355 ...
I0708 20:42:25.849411 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355
I0708 20:42:25.849419 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19195-5988/.minikube/machines
I0708 20:42:25.849430 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19195-5988/.minikube
I0708 20:42:25.849444 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19195-5988
I0708 20:42:25.849463 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0708 20:42:25.849498 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355 (perms=drwx------)
I0708 20:42:25.849508 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home/jenkins
I0708 20:42:25.849521 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins/minikube-integration/19195-5988/.minikube/machines (perms=drwxr-xr-x)
I0708 20:42:25.849531 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins/minikube-integration/19195-5988/.minikube (perms=drwxr-xr-x)
I0708 20:42:25.849546 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins/minikube-integration/19195-5988 (perms=drwxrwxr-x)
I0708 20:42:25.849559 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0708 20:42:25.849576 49548 main.go:141] libmachine: (old-k8s-version-914355) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0708 20:42:25.849589 49548 main.go:141] libmachine: (old-k8s-version-914355) Creating domain...
I0708 20:42:25.849602 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Checking permissions on dir: /home
I0708 20:42:25.849623 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Skipping /home - not owner
I0708 20:42:25.850762 49548 main.go:141] libmachine: (old-k8s-version-914355) define libvirt domain using xml:
I0708 20:42:25.850785 49548 main.go:141] libmachine: (old-k8s-version-914355) <domain type='kvm'>
I0708 20:42:25.850796 49548 main.go:141] libmachine: (old-k8s-version-914355) <name>old-k8s-version-914355</name>
I0708 20:42:25.850804 49548 main.go:141] libmachine: (old-k8s-version-914355) <memory unit='MiB'>2200</memory>
I0708 20:42:25.850813 49548 main.go:141] libmachine: (old-k8s-version-914355) <vcpu>2</vcpu>
I0708 20:42:25.850825 49548 main.go:141] libmachine: (old-k8s-version-914355) <features>
I0708 20:42:25.850833 49548 main.go:141] libmachine: (old-k8s-version-914355) <acpi/>
I0708 20:42:25.850843 49548 main.go:141] libmachine: (old-k8s-version-914355) <apic/>
I0708 20:42:25.850853 49548 main.go:141] libmachine: (old-k8s-version-914355) <pae/>
I0708 20:42:25.850865 49548 main.go:141] libmachine: (old-k8s-version-914355)
I0708 20:42:25.850873 49548 main.go:141] libmachine: (old-k8s-version-914355) </features>
I0708 20:42:25.850880 49548 main.go:141] libmachine: (old-k8s-version-914355) <cpu mode='host-passthrough'>
I0708 20:42:25.850888 49548 main.go:141] libmachine: (old-k8s-version-914355)
I0708 20:42:25.850892 49548 main.go:141] libmachine: (old-k8s-version-914355) </cpu>
I0708 20:42:25.850900 49548 main.go:141] libmachine: (old-k8s-version-914355) <os>
I0708 20:42:25.850905 49548 main.go:141] libmachine: (old-k8s-version-914355) <type>hvm</type>
I0708 20:42:25.850911 49548 main.go:141] libmachine: (old-k8s-version-914355) <boot dev='cdrom'/>
I0708 20:42:25.850915 49548 main.go:141] libmachine: (old-k8s-version-914355) <boot dev='hd'/>
I0708 20:42:25.850923 49548 main.go:141] libmachine: (old-k8s-version-914355) <bootmenu enable='no'/>
I0708 20:42:25.850927 49548 main.go:141] libmachine: (old-k8s-version-914355) </os>
I0708 20:42:25.850941 49548 main.go:141] libmachine: (old-k8s-version-914355) <devices>
I0708 20:42:25.850948 49548 main.go:141] libmachine: (old-k8s-version-914355) <disk type='file' device='cdrom'>
I0708 20:42:25.850957 49548 main.go:141] libmachine: (old-k8s-version-914355) <source file='/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/boot2docker.iso'/>
I0708 20:42:25.850967 49548 main.go:141] libmachine: (old-k8s-version-914355) <target dev='hdc' bus='scsi'/>
I0708 20:42:25.850996 49548 main.go:141] libmachine: (old-k8s-version-914355) <readonly/>
I0708 20:42:25.851018 49548 main.go:141] libmachine: (old-k8s-version-914355) </disk>
I0708 20:42:25.851031 49548 main.go:141] libmachine: (old-k8s-version-914355) <disk type='file' device='disk'>
I0708 20:42:25.851045 49548 main.go:141] libmachine: (old-k8s-version-914355) <driver name='qemu' type='raw' cache='default' io='threads' />
I0708 20:42:25.851069 49548 main.go:141] libmachine: (old-k8s-version-914355) <source file='/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/old-k8s-version-914355.rawdisk'/>
I0708 20:42:25.851080 49548 main.go:141] libmachine: (old-k8s-version-914355) <target dev='hda' bus='virtio'/>
I0708 20:42:25.851104 49548 main.go:141] libmachine: (old-k8s-version-914355) </disk>
I0708 20:42:25.851126 49548 main.go:141] libmachine: (old-k8s-version-914355) <interface type='network'>
I0708 20:42:25.851142 49548 main.go:141] libmachine: (old-k8s-version-914355) <source network='mk-old-k8s-version-914355'/>
I0708 20:42:25.851150 49548 main.go:141] libmachine: (old-k8s-version-914355) <model type='virtio'/>
I0708 20:42:25.851157 49548 main.go:141] libmachine: (old-k8s-version-914355) </interface>
I0708 20:42:25.851170 49548 main.go:141] libmachine: (old-k8s-version-914355) <interface type='network'>
I0708 20:42:25.851181 49548 main.go:141] libmachine: (old-k8s-version-914355) <source network='default'/>
I0708 20:42:25.851193 49548 main.go:141] libmachine: (old-k8s-version-914355) <model type='virtio'/>
I0708 20:42:25.851213 49548 main.go:141] libmachine: (old-k8s-version-914355) </interface>
I0708 20:42:25.851225 49548 main.go:141] libmachine: (old-k8s-version-914355) <serial type='pty'>
I0708 20:42:25.851238 49548 main.go:141] libmachine: (old-k8s-version-914355) <target port='0'/>
I0708 20:42:25.851247 49548 main.go:141] libmachine: (old-k8s-version-914355) </serial>
I0708 20:42:25.851256 49548 main.go:141] libmachine: (old-k8s-version-914355) <console type='pty'>
I0708 20:42:25.851267 49548 main.go:141] libmachine: (old-k8s-version-914355) <target type='serial' port='0'/>
I0708 20:42:25.851279 49548 main.go:141] libmachine: (old-k8s-version-914355) </console>
I0708 20:42:25.851289 49548 main.go:141] libmachine: (old-k8s-version-914355) <rng model='virtio'>
I0708 20:42:25.851299 49548 main.go:141] libmachine: (old-k8s-version-914355) <backend model='random'>/dev/random</backend>
I0708 20:42:25.851310 49548 main.go:141] libmachine: (old-k8s-version-914355) </rng>
I0708 20:42:25.851326 49548 main.go:141] libmachine: (old-k8s-version-914355)
I0708 20:42:25.851337 49548 main.go:141] libmachine: (old-k8s-version-914355)
I0708 20:42:25.851349 49548 main.go:141] libmachine: (old-k8s-version-914355) </devices>
I0708 20:42:25.851359 49548 main.go:141] libmachine: (old-k8s-version-914355) </domain>
I0708 20:42:25.851369 49548 main.go:141] libmachine: (old-k8s-version-914355)
I0708 20:42:25.857853 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:82:e8:be in network default
I0708 20:42:25.858362 49548 main.go:141] libmachine: (old-k8s-version-914355) Ensuring networks are active...
I0708 20:42:25.858387 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:25.859124 49548 main.go:141] libmachine: (old-k8s-version-914355) Ensuring network default is active
I0708 20:42:25.859395 49548 main.go:141] libmachine: (old-k8s-version-914355) Ensuring network mk-old-k8s-version-914355 is active
I0708 20:42:25.859850 49548 main.go:141] libmachine: (old-k8s-version-914355) Getting domain xml...
I0708 20:42:25.860485 49548 main.go:141] libmachine: (old-k8s-version-914355) Creating domain...
I0708 20:42:27.102045 49548 main.go:141] libmachine: (old-k8s-version-914355) Waiting to get IP...
I0708 20:42:27.102870 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:27.103297 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:27.103338 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:27.103254 49893 retry.go:31] will retry after 249.892984ms: waiting for machine to come up
I0708 20:42:27.354786 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:27.355340 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:27.355370 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:27.355299 49893 retry.go:31] will retry after 358.773579ms: waiting for machine to come up
I0708 20:42:27.715911 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:27.716291 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:27.716323 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:27.716256 49893 retry.go:31] will retry after 431.539678ms: waiting for machine to come up
I0708 20:42:28.149705 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:28.150206 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:28.150231 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:28.150172 49893 retry.go:31] will retry after 551.844202ms: waiting for machine to come up
I0708 20:42:28.704026 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:28.704497 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:28.704526 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:28.704449 49893 retry.go:31] will retry after 577.352164ms: waiting for machine to come up
I0708 20:42:29.283021 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:29.283546 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:29.283574 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:29.283492 49893 retry.go:31] will retry after 793.05533ms: waiting for machine to come up
I0708 20:42:30.079510 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:30.079927 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:30.079948 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:30.079870 49893 retry.go:31] will retry after 1.101736353s: waiting for machine to come up
I0708 20:42:31.183439 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:31.183843 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:31.183872 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:31.183822 49893 retry.go:31] will retry after 1.402564728s: waiting for machine to come up
I0708 20:42:32.588423 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:32.588920 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:32.588953 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:32.588891 49893 retry.go:31] will retry after 1.70885095s: waiting for machine to come up
I0708 20:42:34.299955 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:34.300495 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:34.300525 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:34.300445 49893 retry.go:31] will retry after 1.614081319s: waiting for machine to come up
I0708 20:42:35.916022 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:35.916535 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:35.916569 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:35.916450 49893 retry.go:31] will retry after 2.187541225s: waiting for machine to come up
I0708 20:42:38.106413 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:38.106973 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:38.107001 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:38.106928 49893 retry.go:31] will retry after 3.254816979s: waiting for machine to come up
I0708 20:42:41.363037 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:41.363506 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:41.363535 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:41.363425 49893 retry.go:31] will retry after 3.336926102s: waiting for machine to come up
I0708 20:42:44.703946 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:44.704352 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find current IP address of domain old-k8s-version-914355 in network mk-old-k8s-version-914355
I0708 20:42:44.704377 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | I0708 20:42:44.704326 49893 retry.go:31] will retry after 3.830446185s: waiting for machine to come up
I0708 20:42:48.538767 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.539382 49548 main.go:141] libmachine: (old-k8s-version-914355) Found IP for machine: 192.168.50.65
I0708 20:42:48.539405 49548 main.go:141] libmachine: (old-k8s-version-914355) Reserving static IP address...
I0708 20:42:48.539418 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has current primary IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.539758 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-914355", mac: "52:54:00:2b:81:07", ip: "192.168.50.65"} in network mk-old-k8s-version-914355
I0708 20:42:48.615094 49548 main.go:141] libmachine: (old-k8s-version-914355) Reserved static IP address: 192.168.50.65
I0708 20:42:48.615127 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Getting to WaitForSSH function...
I0708 20:42:48.615138 49548 main.go:141] libmachine: (old-k8s-version-914355) Waiting for SSH to be available...
I0708 20:42:48.617629 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.617950 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:minikube Clientid:01:52:54:00:2b:81:07}
I0708 20:42:48.617971 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.618119 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Using SSH client type: external
I0708 20:42:48.618144 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Using SSH private key: /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa (-rw-------)
I0708 20:42:48.618215 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.50.65 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa -p 22] /usr/bin/ssh <nil>}
I0708 20:42:48.618234 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | About to run SSH command:
I0708 20:42:48.618253 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | exit 0
I0708 20:42:48.747881 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | SSH cmd err, output: <nil>:
I0708 20:42:48.748171 49548 main.go:141] libmachine: (old-k8s-version-914355) KVM machine creation complete!
I0708 20:42:48.748480 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetConfigRaw
I0708 20:42:48.749109 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:48.749290 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:48.749472 49548 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0708 20:42:48.749489 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetState
I0708 20:42:48.750651 49548 main.go:141] libmachine: Detecting operating system of created instance...
I0708 20:42:48.750671 49548 main.go:141] libmachine: Waiting for SSH to be available...
I0708 20:42:48.750677 49548 main.go:141] libmachine: Getting to WaitForSSH function...
I0708 20:42:48.750683 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:48.752833 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.753192 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:48.753218 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.753307 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:48.753468 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.753626 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.753803 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:48.753975 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:48.754227 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:48.754244 49548 main.go:141] libmachine: About to run SSH command:
exit 0
I0708 20:42:48.862999 49548 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0708 20:42:48.863022 49548 main.go:141] libmachine: Detecting the provisioner...
I0708 20:42:48.863029 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:48.865728 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.866095 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:48.866121 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.866322 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:48.866564 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.866728 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.866896 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:48.867101 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:48.867280 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:48.867292 49548 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0708 20:42:48.980282 49548 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0708 20:42:48.980357 49548 main.go:141] libmachine: found compatible host: buildroot
I0708 20:42:48.980366 49548 main.go:141] libmachine: Provisioning with buildroot...
I0708 20:42:48.980373 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetMachineName
I0708 20:42:48.980655 49548 buildroot.go:166] provisioning hostname "old-k8s-version-914355"
I0708 20:42:48.980680 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetMachineName
I0708 20:42:48.980874 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:48.983703 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.984050 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:48.984078 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:48.984192 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:48.984380 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.984565 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:48.984716 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:48.984916 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:48.985202 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:48.985233 49548 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-914355 && echo "old-k8s-version-914355" | sudo tee /etc/hostname
I0708 20:42:49.111669 49548 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-914355
I0708 20:42:49.111692 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.114378 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.114708 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.114738 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.114880 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:49.115068 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.115252 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.115409 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:49.115589 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:49.115810 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:49.115839 49548 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-914355' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-914355/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-914355' | sudo tee -a /etc/hosts;
fi
fi
I0708 20:42:49.237458 49548 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0708 20:42:49.237484 49548 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19195-5988/.minikube CaCertPath:/home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19195-5988/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19195-5988/.minikube}
I0708 20:42:49.237509 49548 buildroot.go:174] setting up certificates
I0708 20:42:49.237521 49548 provision.go:84] configureAuth start
I0708 20:42:49.237529 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetMachineName
I0708 20:42:49.237865 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetIP
I0708 20:42:49.240401 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.240769 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.240796 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.240894 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.243053 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.243397 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.243424 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.243571 49548 provision.go:143] copyHostCerts
I0708 20:42:49.243625 49548 exec_runner.go:144] found /home/jenkins/minikube-integration/19195-5988/.minikube/ca.pem, removing ...
I0708 20:42:49.243633 49548 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19195-5988/.minikube/ca.pem
I0708 20:42:49.243687 49548 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19195-5988/.minikube/ca.pem (1078 bytes)
I0708 20:42:49.243781 49548 exec_runner.go:144] found /home/jenkins/minikube-integration/19195-5988/.minikube/cert.pem, removing ...
I0708 20:42:49.243789 49548 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19195-5988/.minikube/cert.pem
I0708 20:42:49.243807 49548 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19195-5988/.minikube/cert.pem (1123 bytes)
I0708 20:42:49.243878 49548 exec_runner.go:144] found /home/jenkins/minikube-integration/19195-5988/.minikube/key.pem, removing ...
I0708 20:42:49.243885 49548 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19195-5988/.minikube/key.pem
I0708 20:42:49.243901 49548 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19195-5988/.minikube/key.pem (1679 bytes)
I0708 20:42:49.243975 49548 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19195-5988/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-914355 san=[127.0.0.1 192.168.50.65 localhost minikube old-k8s-version-914355]
I0708 20:42:49.301351 49548 provision.go:177] copyRemoteCerts
I0708 20:42:49.301400 49548 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0708 20:42:49.301421 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.304132 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.304448 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.304471 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.304715 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:49.304898 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.305032 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:49.305143 49548 sshutil.go:53] new ssh client: &{IP:192.168.50.65 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa Username:docker}
I0708 20:42:49.390497 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0708 20:42:49.418801 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0708 20:42:49.443683 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0708 20:42:49.468129 49548 provision.go:87] duration metric: took 230.597717ms to configureAuth
I0708 20:42:49.468164 49548 buildroot.go:189] setting minikube options for container-runtime
I0708 20:42:49.468334 49548 config.go:182] Loaded profile config "old-k8s-version-914355": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0708 20:42:49.468396 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.471137 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.471481 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.471508 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.471686 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:49.471889 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.472085 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.472239 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:49.472434 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:49.472636 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:49.472652 49548 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0708 20:42:49.746077 49548 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0708 20:42:49.746110 49548 main.go:141] libmachine: Checking connection to Docker...
I0708 20:42:49.746137 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetURL
I0708 20:42:49.747548 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | Using libvirt version 6000000
I0708 20:42:49.749755 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.750103 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.750128 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.750346 49548 main.go:141] libmachine: Docker is up and running!
I0708 20:42:49.750361 49548 main.go:141] libmachine: Reticulating splines...
I0708 20:42:49.750368 49548 client.go:171] duration metric: took 24.303285291s to LocalClient.Create
I0708 20:42:49.750394 49548 start.go:167] duration metric: took 24.303346535s to libmachine.API.Create "old-k8s-version-914355"
I0708 20:42:49.750406 49548 start.go:293] postStartSetup for "old-k8s-version-914355" (driver="kvm2")
I0708 20:42:49.750418 49548 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0708 20:42:49.750442 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:49.750735 49548 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0708 20:42:49.750761 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.752945 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.753288 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.753313 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.753439 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:49.753655 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.753779 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:49.753914 49548 sshutil.go:53] new ssh client: &{IP:192.168.50.65 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa Username:docker}
I0708 20:42:49.842461 49548 ssh_runner.go:195] Run: cat /etc/os-release
I0708 20:42:49.847510 49548 info.go:137] Remote host: Buildroot 2023.02.9
I0708 20:42:49.847540 49548 filesync.go:126] Scanning /home/jenkins/minikube-integration/19195-5988/.minikube/addons for local assets ...
I0708 20:42:49.847597 49548 filesync.go:126] Scanning /home/jenkins/minikube-integration/19195-5988/.minikube/files for local assets ...
I0708 20:42:49.847668 49548 filesync.go:149] local asset: /home/jenkins/minikube-integration/19195-5988/.minikube/files/etc/ssl/certs/131412.pem -> 131412.pem in /etc/ssl/certs
I0708 20:42:49.847757 49548 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0708 20:42:49.857753 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/files/etc/ssl/certs/131412.pem --> /etc/ssl/certs/131412.pem (1708 bytes)
I0708 20:42:49.883444 49548 start.go:296] duration metric: took 133.025545ms for postStartSetup
I0708 20:42:49.883513 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetConfigRaw
I0708 20:42:49.884099 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetIP
I0708 20:42:49.886672 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.886987 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.887007 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.887325 49548 profile.go:143] Saving config to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/config.json ...
I0708 20:42:49.887593 49548 start.go:128] duration metric: took 24.462779193s to createHost
I0708 20:42:49.887620 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:49.889899 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.890261 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:49.890288 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:49.890430 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:49.890608 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.890774 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:49.890881 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:49.891001 49548 main.go:141] libmachine: Using SSH client type: native
I0708 20:42:49.891205 49548 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d980] 0x8306e0 <nil> [] 0s} 192.168.50.65 22 <nil> <nil>}
I0708 20:42:49.891217 49548 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0708 20:42:50.004372 49548 main.go:141] libmachine: SSH cmd err, output: <nil>: 1720471369.979460236
I0708 20:42:50.004392 49548 fix.go:216] guest clock: 1720471369.979460236
I0708 20:42:50.004399 49548 fix.go:229] Guest: 2024-07-08 20:42:49.979460236 +0000 UTC Remote: 2024-07-08 20:42:49.887607855 +0000 UTC m=+50.285135825 (delta=91.852381ms)
I0708 20:42:50.004425 49548 fix.go:200] guest clock delta is within tolerance: 91.852381ms
I0708 20:42:50.004430 49548 start.go:83] releasing machines lock for "old-k8s-version-914355", held for 24.579769451s
I0708 20:42:50.004452 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:50.004719 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetIP
I0708 20:42:50.007608 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.008057 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:50.008087 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.008235 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:50.008891 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:50.009078 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .DriverName
I0708 20:42:50.009171 49548 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0708 20:42:50.009212 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:50.009328 49548 ssh_runner.go:195] Run: cat /version.json
I0708 20:42:50.009354 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHHostname
I0708 20:42:50.011907 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.012096 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.012252 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:50.012280 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.012390 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:50.012413 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:50.012534 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:50.012616 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHPort
I0708 20:42:50.012710 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:50.012769 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHKeyPath
I0708 20:42:50.012877 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:50.012945 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetSSHUsername
I0708 20:42:50.013069 49548 sshutil.go:53] new ssh client: &{IP:192.168.50.65 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa Username:docker}
I0708 20:42:50.013084 49548 sshutil.go:53] new ssh client: &{IP:192.168.50.65 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19195-5988/.minikube/machines/old-k8s-version-914355/id_rsa Username:docker}
I0708 20:42:50.100971 49548 ssh_runner.go:195] Run: systemctl --version
I0708 20:42:50.124527 49548 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0708 20:42:50.295483 49548 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0708 20:42:50.304328 49548 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0708 20:42:50.304398 49548 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0708 20:42:50.334102 49548 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0708 20:42:50.334130 49548 start.go:494] detecting cgroup driver to use...
I0708 20:42:50.334193 49548 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0708 20:42:50.357335 49548 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0708 20:42:50.375483 49548 docker.go:217] disabling cri-docker service (if available) ...
I0708 20:42:50.375556 49548 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0708 20:42:50.391624 49548 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0708 20:42:50.406680 49548 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0708 20:42:50.531672 49548 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0708 20:42:50.675426 49548 docker.go:233] disabling docker service ...
I0708 20:42:50.675535 49548 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0708 20:42:50.690778 49548 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0708 20:42:50.704923 49548 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0708 20:42:50.852221 49548 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0708 20:42:50.978546 49548 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0708 20:42:50.993085 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0708 20:42:51.012824 49548 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0708 20:42:51.012899 49548 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0708 20:42:51.024126 49548 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0708 20:42:51.024223 49548 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0708 20:42:51.035039 49548 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0708 20:42:51.045885 49548 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0708 20:42:51.057016 49548 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0708 20:42:51.067751 49548 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0708 20:42:51.077198 49548 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0708 20:42:51.077249 49548 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0708 20:42:51.090550 49548 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0708 20:42:51.099907 49548 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0708 20:42:51.246988 49548 ssh_runner.go:195] Run: sudo systemctl restart crio
I0708 20:42:51.394500 49548 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0708 20:42:51.394568 49548 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0708 20:42:51.399389 49548 start.go:562] Will wait 60s for crictl version
I0708 20:42:51.399465 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:51.403103 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0708 20:42:51.445367 49548 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0708 20:42:51.445459 49548 ssh_runner.go:195] Run: crio --version
I0708 20:42:51.474316 49548 ssh_runner.go:195] Run: crio --version
I0708 20:42:51.507594 49548 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0708 20:42:51.508747 49548 main.go:141] libmachine: (old-k8s-version-914355) Calling .GetIP
I0708 20:42:51.511720 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:51.512220 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:2b:81:07", ip: ""} in network mk-old-k8s-version-914355: {Iface:virbr2 ExpiryTime:2024-07-08 21:42:39 +0000 UTC Type:0 Mac:52:54:00:2b:81:07 Iaid: IPaddr:192.168.50.65 Prefix:24 Hostname:old-k8s-version-914355 Clientid:01:52:54:00:2b:81:07}
I0708 20:42:51.512330 49548 main.go:141] libmachine: (old-k8s-version-914355) DBG | domain old-k8s-version-914355 has defined IP address 192.168.50.65 and MAC address 52:54:00:2b:81:07 in network mk-old-k8s-version-914355
I0708 20:42:51.513613 49548 ssh_runner.go:195] Run: grep 192.168.50.1 host.minikube.internal$ /etc/hosts
I0708 20:42:51.518047 49548 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.50.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0708 20:42:51.531104 49548 kubeadm.go:877] updating cluster {Name:old-k8s-version-914355 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19186/minikube-v1.33.1-1720011972-19186-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1720012048-19186@sha256:0fc826bca29cbb5a8335aaf40b2c19a34f0a6b85133ca47842ce6e575d3bc2ef Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-914355 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.65 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0708 20:42:51.531233 49548 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0708 20:42:51.531304 49548 ssh_runner.go:195] Run: sudo crictl images --output json
I0708 20:42:51.563806 49548 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0708 20:42:51.563887 49548 ssh_runner.go:195] Run: which lz4
I0708 20:42:51.568121 49548 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0708 20:42:51.572424 49548 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0708 20:42:51.572463 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0708 20:42:53.377051 49548 crio.go:462] duration metric: took 1.808954524s to copy over tarball
I0708 20:42:53.377147 49548 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0708 20:42:56.100761 49548 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.723587759s)
I0708 20:42:56.100786 49548 crio.go:469] duration metric: took 2.723698222s to extract the tarball
I0708 20:42:56.100795 49548 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0708 20:42:56.144012 49548 ssh_runner.go:195] Run: sudo crictl images --output json
I0708 20:42:56.195420 49548 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0708 20:42:56.195478 49548 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0708 20:42:56.195570 49548 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0708 20:42:56.195570 49548 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0708 20:42:56.195628 49548 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0708 20:42:56.195701 49548 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0708 20:42:56.195711 49548 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0708 20:42:56.195724 49548 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0708 20:42:56.195909 49548 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0708 20:42:56.195575 49548 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0708 20:42:56.197883 49548 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0708 20:42:56.197943 49548 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0708 20:42:56.197959 49548 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0708 20:42:56.198054 49548 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0708 20:42:56.198221 49548 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0708 20:42:56.198315 49548 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0708 20:42:56.198383 49548 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0708 20:42:56.198479 49548 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0708 20:42:56.363014 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0708 20:42:56.364532 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0708 20:42:56.366867 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0708 20:42:56.375227 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0708 20:42:56.383467 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0708 20:42:56.390103 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0708 20:42:56.433607 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0708 20:42:56.474024 49548 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0708 20:42:56.474074 49548 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0708 20:42:56.474148 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.494360 49548 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0708 20:42:56.594131 49548 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0708 20:42:56.594195 49548 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0708 20:42:56.594265 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.594528 49548 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0708 20:42:56.594564 49548 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0708 20:42:56.594609 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.656054 49548 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0708 20:42:56.656108 49548 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0708 20:42:56.656122 49548 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0708 20:42:56.656156 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.656163 49548 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0708 20:42:56.656199 49548 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0708 20:42:56.656217 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0708 20:42:56.656228 49548 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0708 20:42:56.656205 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.656273 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.656369 49548 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0708 20:42:56.656401 49548 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0708 20:42:56.656428 49548 ssh_runner.go:195] Run: which crictl
I0708 20:42:56.750305 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0708 20:42:56.750339 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0708 20:42:56.750369 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0708 20:42:56.750416 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0708 20:42:56.750479 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0708 20:42:56.750480 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0708 20:42:56.750531 49548 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0708 20:42:56.895197 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0708 20:42:56.895201 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0708 20:42:56.895233 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0708 20:42:56.895280 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0708 20:42:56.895350 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0708 20:42:56.895366 49548 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0708 20:42:56.895407 49548 cache_images.go:92] duration metric: took 699.908872ms to LoadCachedImages
W0708 20:42:56.895504 49548 out.go:239] X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/19195-5988/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0708 20:42:56.895520 49548 kubeadm.go:928] updating node { 192.168.50.65 8443 v1.20.0 crio true true} ...
I0708 20:42:56.895626 49548 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-914355 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.50.65
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-914355 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0708 20:42:56.895697 49548 ssh_runner.go:195] Run: crio config
I0708 20:42:56.947882 49548 cni.go:84] Creating CNI manager for ""
I0708 20:42:56.947906 49548 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0708 20:42:56.947917 49548 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0708 20:42:56.947939 49548 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.50.65 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-914355 NodeName:old-k8s-version-914355 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.50.65"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.50.65 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0708 20:42:56.948136 49548 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.50.65
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-914355"
kubeletExtraArgs:
node-ip: 192.168.50.65
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.50.65"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0708 20:42:56.948209 49548 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0708 20:42:56.958425 49548 binaries.go:44] Found k8s binaries, skipping transfer
I0708 20:42:56.958518 49548 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0708 20:42:56.968850 49548 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0708 20:42:56.988974 49548 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0708 20:42:57.006757 49548 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0708 20:42:57.024356 49548 ssh_runner.go:195] Run: grep 192.168.50.65 control-plane.minikube.internal$ /etc/hosts
I0708 20:42:57.028341 49548 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.50.65 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0708 20:42:57.043488 49548 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0708 20:42:57.182615 49548 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0708 20:42:57.203459 49548 certs.go:68] Setting up /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355 for IP: 192.168.50.65
I0708 20:42:57.203479 49548 certs.go:194] generating shared ca certs ...
I0708 20:42:57.203499 49548 certs.go:226] acquiring lock for ca certs: {Name:mk2b44e1aac6acd08d7e818dc0b4a0d63e10f898 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.203666 49548 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19195-5988/.minikube/ca.key
I0708 20:42:57.203711 49548 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19195-5988/.minikube/proxy-client-ca.key
I0708 20:42:57.203724 49548 certs.go:256] generating profile certs ...
I0708 20:42:57.203788 49548 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.key
I0708 20:42:57.203807 49548 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.crt with IP's: []
I0708 20:42:57.440280 49548 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.crt ...
I0708 20:42:57.440309 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.crt: {Name:mk5ee4bafa2efd5e3315abd3a8f2fde36349f853 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.440490 49548 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.key ...
I0708 20:42:57.440507 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/client.key: {Name:mk4a3080546a28b70f60e69509ab7f2b767fab65 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.440604 49548 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key.8b45f3cf
I0708 20:42:57.440625 49548 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt.8b45f3cf with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.50.65]
I0708 20:42:57.547903 49548 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt.8b45f3cf ...
I0708 20:42:57.547928 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt.8b45f3cf: {Name:mk985889685bc048e5de4e139c5e0e204642578c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.573911 49548 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key.8b45f3cf ...
I0708 20:42:57.573946 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key.8b45f3cf: {Name:mk9c23c12f25bc45248a46018eb3a7e5069d2eb4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.574099 49548 certs.go:381] copying /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt.8b45f3cf -> /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt
I0708 20:42:57.574202 49548 certs.go:385] copying /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key.8b45f3cf -> /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key
I0708 20:42:57.574289 49548 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.key
I0708 20:42:57.574314 49548 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.crt with IP's: []
I0708 20:42:57.669852 49548 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.crt ...
I0708 20:42:57.669890 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.crt: {Name:mk57b71a28602b45f8aeaac279fd5fdd91885fa2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.672809 49548 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.key ...
I0708 20:42:57.672834 49548 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.key: {Name:mkb7adce6c84f5a933251defb6dd4886bfbffb9d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0708 20:42:57.673089 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/13141.pem (1338 bytes)
W0708 20:42:57.673129 49548 certs.go:480] ignoring /home/jenkins/minikube-integration/19195-5988/.minikube/certs/13141_empty.pem, impossibly tiny 0 bytes
I0708 20:42:57.673138 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca-key.pem (1679 bytes)
I0708 20:42:57.673169 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/ca.pem (1078 bytes)
I0708 20:42:57.673198 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/cert.pem (1123 bytes)
I0708 20:42:57.673222 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/certs/key.pem (1679 bytes)
I0708 20:42:57.673276 49548 certs.go:484] found cert: /home/jenkins/minikube-integration/19195-5988/.minikube/files/etc/ssl/certs/131412.pem (1708 bytes)
I0708 20:42:57.674102 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0708 20:42:57.703925 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0708 20:42:57.738922 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0708 20:42:57.772060 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0708 20:42:57.806293 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0708 20:42:57.832407 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0708 20:42:57.863226 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0708 20:42:57.894698 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/profiles/old-k8s-version-914355/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0708 20:42:57.926708 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0708 20:42:57.952023 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/certs/13141.pem --> /usr/share/ca-certificates/13141.pem (1338 bytes)
I0708 20:42:57.980678 49548 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19195-5988/.minikube/files/etc/ssl/certs/131412.pem --> /usr/share/ca-certificates/131412.pem (1708 bytes)
I0708 20:42:58.010345 49548 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0708 20:42:58.043220 49548 ssh_runner.go:195] Run: openssl version
I0708 20:42:58.052769 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/13141.pem && ln -fs /usr/share/ca-certificates/13141.pem /etc/ssl/certs/13141.pem"
I0708 20:42:58.072686 49548 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/13141.pem
I0708 20:42:58.078418 49548 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jul 8 19:41 /usr/share/ca-certificates/13141.pem
I0708 20:42:58.078490 49548 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/13141.pem
I0708 20:42:58.088283 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/13141.pem /etc/ssl/certs/51391683.0"
I0708 20:42:58.108668 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/131412.pem && ln -fs /usr/share/ca-certificates/131412.pem /etc/ssl/certs/131412.pem"
I0708 20:42:58.120345 49548 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/131412.pem
I0708 20:42:58.126237 49548 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jul 8 19:41 /usr/share/ca-certificates/131412.pem
I0708 20:42:58.126302 49548 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/131412.pem
I0708 20:42:58.132187 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/131412.pem /etc/ssl/certs/3ec20f2e.0"
I0708 20:42:58.146509 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0708 20:42:58.158331 49548 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0708 20:42:58.163111 49548 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jul 8 19:29 /usr/share/ca-certificates/minikubeCA.pem
I0708 20:42:58.163166 49548 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0708 20:42:58.169554 49548 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0708 20:42:58.180923 49548 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0708 20:42:58.185235 49548 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0708 20:42:58.185304 49548 kubeadm.go:391] StartCluster: {Name:old-k8s-version-914355 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19186/minikube-v1.33.1-1720011972-19186-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1720012048-19186@sha256:0fc826bca29cbb5a8335aaf40b2c19a34f0a6b85133ca47842ce6e575d3bc2ef Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-914355 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.65 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0708 20:42:58.185414 49548 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0708 20:42:58.185470 49548 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0708 20:42:58.223466 49548 cri.go:89] found id: ""
I0708 20:42:58.223543 49548 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0708 20:42:58.233824 49548 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0708 20:42:58.243946 49548 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0708 20:42:58.254849 49548 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0708 20:42:58.254870 49548 kubeadm.go:156] found existing configuration files:
I0708 20:42:58.254921 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0708 20:42:58.264304 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0708 20:42:58.264374 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0708 20:42:58.274292 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0708 20:42:58.283289 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0708 20:42:58.283353 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0708 20:42:58.292750 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0708 20:42:58.303733 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0708 20:42:58.303794 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0708 20:42:58.315260 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0708 20:42:58.326630 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0708 20:42:58.326701 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0708 20:42:58.338778 49548 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0708 20:42:58.453681 49548 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0708 20:42:58.453766 49548 kubeadm.go:309] [preflight] Running pre-flight checks
I0708 20:42:58.617299 49548 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0708 20:42:58.617447 49548 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0708 20:42:58.617629 49548 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0708 20:42:58.851240 49548 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0708 20:42:58.869203 49548 out.go:204] - Generating certificates and keys ...
I0708 20:42:58.869339 49548 kubeadm.go:309] [certs] Using existing ca certificate authority
I0708 20:42:58.869435 49548 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0708 20:42:59.237213 49548 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0708 20:42:59.350500 49548 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0708 20:42:59.500921 49548 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0708 20:42:59.850576 49548 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0708 20:43:00.047537 49548 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0708 20:43:00.047913 49548 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
I0708 20:43:00.140569 49548 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0708 20:43:00.141092 49548 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
I0708 20:43:00.231791 49548 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0708 20:43:00.326810 49548 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0708 20:43:00.628298 49548 kubeadm.go:309] [certs] Generating "sa" key and public key
I0708 20:43:00.628623 49548 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0708 20:43:00.859430 49548 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0708 20:43:01.136595 49548 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0708 20:43:01.350645 49548 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0708 20:43:01.583446 49548 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0708 20:43:01.603190 49548 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0708 20:43:01.605643 49548 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0708 20:43:01.605802 49548 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0708 20:43:01.758234 49548 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0708 20:43:01.760286 49548 out.go:204] - Booting up control plane ...
I0708 20:43:01.760414 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0708 20:43:01.772063 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0708 20:43:01.774478 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0708 20:43:01.779337 49548 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0708 20:43:01.789994 49548 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0708 20:43:41.785682 49548 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0708 20:43:41.785833 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:43:41.786069 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:43:46.786832 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:43:46.787125 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:43:56.786326 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:43:56.786522 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:44:16.786912 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:44:16.787210 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:44:56.788854 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:44:56.789129 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:44:56.789154 49548 kubeadm.go:309]
I0708 20:44:56.789208 49548 kubeadm.go:309] Unfortunately, an error has occurred:
I0708 20:44:56.789291 49548 kubeadm.go:309] timed out waiting for the condition
I0708 20:44:56.789320 49548 kubeadm.go:309]
I0708 20:44:56.789368 49548 kubeadm.go:309] This error is likely caused by:
I0708 20:44:56.789424 49548 kubeadm.go:309] - The kubelet is not running
I0708 20:44:56.789566 49548 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0708 20:44:56.789578 49548 kubeadm.go:309]
I0708 20:44:56.789850 49548 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0708 20:44:56.789910 49548 kubeadm.go:309] - 'systemctl status kubelet'
I0708 20:44:56.789957 49548 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0708 20:44:56.789967 49548 kubeadm.go:309]
I0708 20:44:56.790112 49548 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0708 20:44:56.790213 49548 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0708 20:44:56.790223 49548 kubeadm.go:309]
I0708 20:44:56.790362 49548 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0708 20:44:56.790487 49548 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0708 20:44:56.790612 49548 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0708 20:44:56.790715 49548 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0708 20:44:56.790728 49548 kubeadm.go:309]
I0708 20:44:56.791596 49548 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0708 20:44:56.791716 49548 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0708 20:44:56.791813 49548 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
W0708 20:44:56.791964 49548 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-914355] and IPs [192.168.50.65 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0708 20:44:56.792023 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0708 20:44:57.301619 49548 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0708 20:44:57.324476 49548 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0708 20:44:57.345873 49548 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0708 20:44:57.345900 49548 kubeadm.go:156] found existing configuration files:
I0708 20:44:57.345954 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0708 20:44:57.359918 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0708 20:44:57.359982 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0708 20:44:57.374581 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0708 20:44:57.388173 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0708 20:44:57.388257 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0708 20:44:57.405746 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0708 20:44:57.426336 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0708 20:44:57.426427 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0708 20:44:57.443309 49548 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0708 20:44:57.458595 49548 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0708 20:44:57.458651 49548 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0708 20:44:57.474730 49548 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0708 20:44:57.571864 49548 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0708 20:44:57.571939 49548 kubeadm.go:309] [preflight] Running pre-flight checks
I0708 20:44:57.755781 49548 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0708 20:44:57.755919 49548 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0708 20:44:57.756040 49548 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0708 20:44:58.002184 49548 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0708 20:44:58.062515 49548 out.go:204] - Generating certificates and keys ...
I0708 20:44:58.062631 49548 kubeadm.go:309] [certs] Using existing ca certificate authority
I0708 20:44:58.062692 49548 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0708 20:44:58.062800 49548 kubeadm.go:309] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0708 20:44:58.062914 49548 kubeadm.go:309] [certs] Using existing front-proxy-ca certificate authority
I0708 20:44:58.063027 49548 kubeadm.go:309] [certs] Using existing front-proxy-client certificate and key on disk
I0708 20:44:58.063102 49548 kubeadm.go:309] [certs] Using existing etcd/ca certificate authority
I0708 20:44:58.063189 49548 kubeadm.go:309] [certs] Using existing etcd/server certificate and key on disk
I0708 20:44:58.063281 49548 kubeadm.go:309] [certs] Using existing etcd/peer certificate and key on disk
I0708 20:44:58.063371 49548 kubeadm.go:309] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0708 20:44:58.063481 49548 kubeadm.go:309] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0708 20:44:58.063549 49548 kubeadm.go:309] [certs] Using the existing "sa" key
I0708 20:44:58.063627 49548 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0708 20:44:58.063698 49548 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0708 20:44:58.159631 49548 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0708 20:44:58.378949 49548 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0708 20:44:58.531648 49548 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0708 20:44:58.555187 49548 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0708 20:44:58.556499 49548 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0708 20:44:58.556563 49548 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0708 20:44:58.700344 49548 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0708 20:44:58.702216 49548 out.go:204] - Booting up control plane ...
I0708 20:44:58.702343 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0708 20:44:58.706660 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0708 20:44:58.707710 49548 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0708 20:44:58.708505 49548 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0708 20:44:58.710757 49548 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0708 20:45:38.714861 49548 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0708 20:45:38.715124 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:45:38.715421 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:45:43.715777 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:45:43.716020 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:45:53.716805 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:45:53.717092 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:46:13.715957 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:46:13.716266 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:46:53.715783 49548 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0708 20:46:53.716078 49548 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0708 20:46:53.716103 49548 kubeadm.go:309]
I0708 20:46:53.716169 49548 kubeadm.go:309] Unfortunately, an error has occurred:
I0708 20:46:53.716224 49548 kubeadm.go:309] timed out waiting for the condition
I0708 20:46:53.716234 49548 kubeadm.go:309]
I0708 20:46:53.716298 49548 kubeadm.go:309] This error is likely caused by:
I0708 20:46:53.716350 49548 kubeadm.go:309] - The kubelet is not running
I0708 20:46:53.716476 49548 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0708 20:46:53.716484 49548 kubeadm.go:309]
I0708 20:46:53.716613 49548 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0708 20:46:53.716654 49548 kubeadm.go:309] - 'systemctl status kubelet'
I0708 20:46:53.716695 49548 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0708 20:46:53.716701 49548 kubeadm.go:309]
I0708 20:46:53.716845 49548 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0708 20:46:53.716956 49548 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0708 20:46:53.716963 49548 kubeadm.go:309]
I0708 20:46:53.717110 49548 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0708 20:46:53.717235 49548 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0708 20:46:53.717328 49548 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0708 20:46:53.717421 49548 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0708 20:46:53.717430 49548 kubeadm.go:309]
I0708 20:46:53.718420 49548 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0708 20:46:53.718526 49548 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0708 20:46:53.718585 49548 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0708 20:46:53.718664 49548 kubeadm.go:393] duration metric: took 3m55.533363646s to StartCluster
I0708 20:46:53.718715 49548 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0708 20:46:53.718778 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0708 20:46:53.798556 49548 cri.go:89] found id: ""
I0708 20:46:53.798591 49548 logs.go:276] 0 containers: []
W0708 20:46:53.798602 49548 logs.go:278] No container was found matching "kube-apiserver"
I0708 20:46:53.798609 49548 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0708 20:46:53.798672 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0708 20:46:53.873765 49548 cri.go:89] found id: ""
I0708 20:46:53.873792 49548 logs.go:276] 0 containers: []
W0708 20:46:53.873802 49548 logs.go:278] No container was found matching "etcd"
I0708 20:46:53.873809 49548 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0708 20:46:53.873860 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0708 20:46:53.924745 49548 cri.go:89] found id: ""
I0708 20:46:53.924767 49548 logs.go:276] 0 containers: []
W0708 20:46:53.924781 49548 logs.go:278] No container was found matching "coredns"
I0708 20:46:53.924788 49548 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0708 20:46:53.924839 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0708 20:46:53.981130 49548 cri.go:89] found id: ""
I0708 20:46:53.981155 49548 logs.go:276] 0 containers: []
W0708 20:46:53.981167 49548 logs.go:278] No container was found matching "kube-scheduler"
I0708 20:46:53.981175 49548 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0708 20:46:53.981242 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0708 20:46:54.057038 49548 cri.go:89] found id: ""
I0708 20:46:54.057065 49548 logs.go:276] 0 containers: []
W0708 20:46:54.057080 49548 logs.go:278] No container was found matching "kube-proxy"
I0708 20:46:54.057087 49548 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0708 20:46:54.057149 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0708 20:46:54.116710 49548 cri.go:89] found id: ""
I0708 20:46:54.116734 49548 logs.go:276] 0 containers: []
W0708 20:46:54.116744 49548 logs.go:278] No container was found matching "kube-controller-manager"
I0708 20:46:54.116752 49548 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0708 20:46:54.116812 49548 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0708 20:46:54.168323 49548 cri.go:89] found id: ""
I0708 20:46:54.168349 49548 logs.go:276] 0 containers: []
W0708 20:46:54.168359 49548 logs.go:278] No container was found matching "kindnet"
I0708 20:46:54.168370 49548 logs.go:123] Gathering logs for CRI-O ...
I0708 20:46:54.168383 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0708 20:46:54.297128 49548 logs.go:123] Gathering logs for container status ...
I0708 20:46:54.297160 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0708 20:46:54.366966 49548 logs.go:123] Gathering logs for kubelet ...
I0708 20:46:54.366999 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0708 20:46:54.446027 49548 logs.go:123] Gathering logs for dmesg ...
I0708 20:46:54.446068 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0708 20:46:54.466178 49548 logs.go:123] Gathering logs for describe nodes ...
I0708 20:46:54.466224 49548 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0708 20:46:54.634729 49548 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
W0708 20:46:54.634801 49548 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0708 20:46:54.634869 49548 out.go:239] *
*
W0708 20:46:54.634935 49548 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0708 20:46:54.634966 49548 out.go:239] *
*
W0708 20:46:54.636037 49548 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0708 20:46:54.639509 49548 out.go:177]
W0708 20:46:54.640777 49548 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0708 20:46:54.640857 49548 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0708 20:46:54.640888 49548 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0708 20:46:54.642393 49548 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-914355 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-914355 -n old-k8s-version-914355
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-914355 -n old-k8s-version-914355: exit status 6 (284.412735ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0708 20:46:54.970568 55831 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-914355" does not appear in /home/jenkins/minikube-integration/19195-5988/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-914355" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (295.39s)