=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-000272 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
E0723 15:09:49.699488 18503 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/addons-566823/client.crt: no such file or directory
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-000272 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m51.059238799s)
-- stdout --
* [old-k8s-version-000272] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19319
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19319-11303/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19319-11303/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-000272" primary control-plane node in "old-k8s-version-000272" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0723 15:09:40.227450 61145 out.go:291] Setting OutFile to fd 1 ...
I0723 15:09:40.227998 61145 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0723 15:09:40.228054 61145 out.go:304] Setting ErrFile to fd 2...
I0723 15:09:40.228072 61145 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0723 15:09:40.228556 61145 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19319-11303/.minikube/bin
I0723 15:09:40.229557 61145 out.go:298] Setting JSON to false
I0723 15:09:40.231063 61145 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":6726,"bootTime":1721740654,"procs":211,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0723 15:09:40.231146 61145 start.go:139] virtualization: kvm guest
I0723 15:09:40.233467 61145 out.go:177] * [old-k8s-version-000272] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0723 15:09:40.234931 61145 out.go:177] - MINIKUBE_LOCATION=19319
I0723 15:09:40.234967 61145 notify.go:220] Checking for updates...
I0723 15:09:40.237447 61145 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0723 15:09:40.238611 61145 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19319-11303/kubeconfig
I0723 15:09:40.239726 61145 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19319-11303/.minikube
I0723 15:09:40.240944 61145 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0723 15:09:40.242094 61145 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0723 15:09:40.243761 61145 config.go:182] Loaded profile config "cert-expiration-457920": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0723 15:09:40.243879 61145 config.go:182] Loaded profile config "kubernetes-upgrade-503350": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0723 15:09:40.243978 61145 config.go:182] Loaded profile config "stopped-upgrade-193974": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.24.1
I0723 15:09:40.244074 61145 driver.go:392] Setting default libvirt URI to qemu:///system
I0723 15:09:40.283441 61145 out.go:177] * Using the kvm2 driver based on user configuration
I0723 15:09:40.284626 61145 start.go:297] selected driver: kvm2
I0723 15:09:40.284642 61145 start.go:901] validating driver "kvm2" against <nil>
I0723 15:09:40.284656 61145 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0723 15:09:40.285385 61145 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0723 15:09:40.285485 61145 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19319-11303/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0723 15:09:40.301561 61145 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0723 15:09:40.301633 61145 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0723 15:09:40.301906 61145 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0723 15:09:40.301978 61145 cni.go:84] Creating CNI manager for ""
I0723 15:09:40.301994 61145 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0723 15:09:40.302002 61145 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0723 15:09:40.302072 61145 start.go:340] cluster config:
{Name:old-k8s-version-000272 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721687125-19319@sha256:8e08c9232f6be53a68fac8937bc5a03a99e38d6a1ff7b6ea4048c989041004ae Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-000272 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0723 15:09:40.302185 61145 iso.go:125] acquiring lock: {Name:mk4b004df17d8bd7e7f5be3e4c1c583053b331d8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0723 15:09:40.303916 61145 out.go:177] * Starting "old-k8s-version-000272" primary control-plane node in "old-k8s-version-000272" cluster
I0723 15:09:40.305127 61145 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0723 15:09:40.305161 61145 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0723 15:09:40.305174 61145 cache.go:56] Caching tarball of preloaded images
I0723 15:09:40.305253 61145 preload.go:172] Found /home/jenkins/minikube-integration/19319-11303/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0723 15:09:40.305267 61145 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0723 15:09:40.305401 61145 profile.go:143] Saving config to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/config.json ...
I0723 15:09:40.305426 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/config.json: {Name:mkdd082cebfc33c5b2db5f82f8f995c1c6c725d7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:09:40.305589 61145 start.go:360] acquireMachinesLock for old-k8s-version-000272: {Name:mkc8ffa8d1a7cde4fb65dc3bdbc209df14d9c326 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0723 15:10:02.874773 61145 start.go:364] duration metric: took 22.569153581s to acquireMachinesLock for "old-k8s-version-000272"
I0723 15:10:02.874855 61145 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-000272 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721687125-19319@sha256:8e08c9232f6be53a68fac8937bc5a03a99e38d6a1ff7b6ea4048c989041004ae Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-000272 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0723 15:10:02.874987 61145 start.go:125] createHost starting for "" (driver="kvm2")
I0723 15:10:02.877170 61145 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0723 15:10:02.877367 61145 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0723 15:10:02.877408 61145 main.go:141] libmachine: Launching plugin server for driver kvm2
I0723 15:10:02.893882 61145 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:42059
I0723 15:10:02.894313 61145 main.go:141] libmachine: () Calling .GetVersion
I0723 15:10:02.894868 61145 main.go:141] libmachine: Using API Version 1
I0723 15:10:02.894889 61145 main.go:141] libmachine: () Calling .SetConfigRaw
I0723 15:10:02.895268 61145 main.go:141] libmachine: () Calling .GetMachineName
I0723 15:10:02.895442 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetMachineName
I0723 15:10:02.895588 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:02.895756 61145 start.go:159] libmachine.API.Create for "old-k8s-version-000272" (driver="kvm2")
I0723 15:10:02.895815 61145 client.go:168] LocalClient.Create starting
I0723 15:10:02.895849 61145 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem
I0723 15:10:02.895896 61145 main.go:141] libmachine: Decoding PEM data...
I0723 15:10:02.895919 61145 main.go:141] libmachine: Parsing certificate...
I0723 15:10:02.895983 61145 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19319-11303/.minikube/certs/cert.pem
I0723 15:10:02.896007 61145 main.go:141] libmachine: Decoding PEM data...
I0723 15:10:02.896024 61145 main.go:141] libmachine: Parsing certificate...
I0723 15:10:02.896059 61145 main.go:141] libmachine: Running pre-create checks...
I0723 15:10:02.896077 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .PreCreateCheck
I0723 15:10:02.896425 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetConfigRaw
I0723 15:10:02.896860 61145 main.go:141] libmachine: Creating machine...
I0723 15:10:02.896879 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .Create
I0723 15:10:02.897010 61145 main.go:141] libmachine: (old-k8s-version-000272) Creating KVM machine...
I0723 15:10:02.898222 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found existing default KVM network
I0723 15:10:02.899667 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:02.899523 61404 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:08:ac:0c} reservation:<nil>}
I0723 15:10:02.900611 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:02.900535 61404 network.go:206] using free private subnet 192.168.50.0/24: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0002a2970}
I0723 15:10:02.900640 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | created network xml:
I0723 15:10:02.900651 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <network>
I0723 15:10:02.900662 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <name>mk-old-k8s-version-000272</name>
I0723 15:10:02.900675 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <dns enable='no'/>
I0723 15:10:02.900685 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG |
I0723 15:10:02.900699 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <ip address='192.168.50.1' netmask='255.255.255.0'>
I0723 15:10:02.900713 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <dhcp>
I0723 15:10:02.900731 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | <range start='192.168.50.2' end='192.168.50.253'/>
I0723 15:10:02.900743 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | </dhcp>
I0723 15:10:02.900754 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | </ip>
I0723 15:10:02.900764 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG |
I0723 15:10:02.900776 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | </network>
I0723 15:10:02.900796 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG |
I0723 15:10:02.906064 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | trying to create private KVM network mk-old-k8s-version-000272 192.168.50.0/24...
I0723 15:10:02.974959 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | private KVM network mk-old-k8s-version-000272 192.168.50.0/24 created
I0723 15:10:02.975006 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting up store path in /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272 ...
I0723 15:10:02.975031 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:02.974914 61404 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19319-11303/.minikube
I0723 15:10:02.975054 61145 main.go:141] libmachine: (old-k8s-version-000272) Building disk image from file:///home/jenkins/minikube-integration/19319-11303/.minikube/cache/iso/amd64/minikube-v1.33.1-1721690939-19319-amd64.iso
I0723 15:10:02.975075 61145 main.go:141] libmachine: (old-k8s-version-000272) Downloading /home/jenkins/minikube-integration/19319-11303/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19319-11303/.minikube/cache/iso/amd64/minikube-v1.33.1-1721690939-19319-amd64.iso...
I0723 15:10:03.218647 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:03.218468 61404 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa...
I0723 15:10:03.260954 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:03.260830 61404 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/old-k8s-version-000272.rawdisk...
I0723 15:10:03.260985 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Writing magic tar header
I0723 15:10:03.260998 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Writing SSH key tar header
I0723 15:10:03.261012 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:03.260958 61404 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272 ...
I0723 15:10:03.261135 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272
I0723 15:10:03.261167 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19319-11303/.minikube/machines
I0723 15:10:03.261180 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272 (perms=drwx------)
I0723 15:10:03.261204 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins/minikube-integration/19319-11303/.minikube/machines (perms=drwxr-xr-x)
I0723 15:10:03.261218 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins/minikube-integration/19319-11303/.minikube (perms=drwxr-xr-x)
I0723 15:10:03.261231 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins/minikube-integration/19319-11303 (perms=drwxrwxr-x)
I0723 15:10:03.261241 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0723 15:10:03.261254 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19319-11303/.minikube
I0723 15:10:03.261265 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19319-11303
I0723 15:10:03.261278 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0723 15:10:03.261318 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home/jenkins
I0723 15:10:03.261363 61145 main.go:141] libmachine: (old-k8s-version-000272) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0723 15:10:03.261378 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Checking permissions on dir: /home
I0723 15:10:03.261386 61145 main.go:141] libmachine: (old-k8s-version-000272) Creating domain...
I0723 15:10:03.261399 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Skipping /home - not owner
I0723 15:10:03.262362 61145 main.go:141] libmachine: (old-k8s-version-000272) define libvirt domain using xml:
I0723 15:10:03.262401 61145 main.go:141] libmachine: (old-k8s-version-000272) <domain type='kvm'>
I0723 15:10:03.262439 61145 main.go:141] libmachine: (old-k8s-version-000272) <name>old-k8s-version-000272</name>
I0723 15:10:03.262463 61145 main.go:141] libmachine: (old-k8s-version-000272) <memory unit='MiB'>2200</memory>
I0723 15:10:03.262472 61145 main.go:141] libmachine: (old-k8s-version-000272) <vcpu>2</vcpu>
I0723 15:10:03.262487 61145 main.go:141] libmachine: (old-k8s-version-000272) <features>
I0723 15:10:03.262499 61145 main.go:141] libmachine: (old-k8s-version-000272) <acpi/>
I0723 15:10:03.262507 61145 main.go:141] libmachine: (old-k8s-version-000272) <apic/>
I0723 15:10:03.262528 61145 main.go:141] libmachine: (old-k8s-version-000272) <pae/>
I0723 15:10:03.262537 61145 main.go:141] libmachine: (old-k8s-version-000272)
I0723 15:10:03.262545 61145 main.go:141] libmachine: (old-k8s-version-000272) </features>
I0723 15:10:03.262556 61145 main.go:141] libmachine: (old-k8s-version-000272) <cpu mode='host-passthrough'>
I0723 15:10:03.262566 61145 main.go:141] libmachine: (old-k8s-version-000272)
I0723 15:10:03.262574 61145 main.go:141] libmachine: (old-k8s-version-000272) </cpu>
I0723 15:10:03.262588 61145 main.go:141] libmachine: (old-k8s-version-000272) <os>
I0723 15:10:03.262607 61145 main.go:141] libmachine: (old-k8s-version-000272) <type>hvm</type>
I0723 15:10:03.262617 61145 main.go:141] libmachine: (old-k8s-version-000272) <boot dev='cdrom'/>
I0723 15:10:03.262629 61145 main.go:141] libmachine: (old-k8s-version-000272) <boot dev='hd'/>
I0723 15:10:03.262637 61145 main.go:141] libmachine: (old-k8s-version-000272) <bootmenu enable='no'/>
I0723 15:10:03.262645 61145 main.go:141] libmachine: (old-k8s-version-000272) </os>
I0723 15:10:03.262650 61145 main.go:141] libmachine: (old-k8s-version-000272) <devices>
I0723 15:10:03.262659 61145 main.go:141] libmachine: (old-k8s-version-000272) <disk type='file' device='cdrom'>
I0723 15:10:03.262679 61145 main.go:141] libmachine: (old-k8s-version-000272) <source file='/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/boot2docker.iso'/>
I0723 15:10:03.262691 61145 main.go:141] libmachine: (old-k8s-version-000272) <target dev='hdc' bus='scsi'/>
I0723 15:10:03.262700 61145 main.go:141] libmachine: (old-k8s-version-000272) <readonly/>
I0723 15:10:03.262709 61145 main.go:141] libmachine: (old-k8s-version-000272) </disk>
I0723 15:10:03.262719 61145 main.go:141] libmachine: (old-k8s-version-000272) <disk type='file' device='disk'>
I0723 15:10:03.262734 61145 main.go:141] libmachine: (old-k8s-version-000272) <driver name='qemu' type='raw' cache='default' io='threads' />
I0723 15:10:03.262763 61145 main.go:141] libmachine: (old-k8s-version-000272) <source file='/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/old-k8s-version-000272.rawdisk'/>
I0723 15:10:03.262778 61145 main.go:141] libmachine: (old-k8s-version-000272) <target dev='hda' bus='virtio'/>
I0723 15:10:03.262785 61145 main.go:141] libmachine: (old-k8s-version-000272) </disk>
I0723 15:10:03.262797 61145 main.go:141] libmachine: (old-k8s-version-000272) <interface type='network'>
I0723 15:10:03.262809 61145 main.go:141] libmachine: (old-k8s-version-000272) <source network='mk-old-k8s-version-000272'/>
I0723 15:10:03.262817 61145 main.go:141] libmachine: (old-k8s-version-000272) <model type='virtio'/>
I0723 15:10:03.262828 61145 main.go:141] libmachine: (old-k8s-version-000272) </interface>
I0723 15:10:03.262837 61145 main.go:141] libmachine: (old-k8s-version-000272) <interface type='network'>
I0723 15:10:03.262849 61145 main.go:141] libmachine: (old-k8s-version-000272) <source network='default'/>
I0723 15:10:03.262861 61145 main.go:141] libmachine: (old-k8s-version-000272) <model type='virtio'/>
I0723 15:10:03.262870 61145 main.go:141] libmachine: (old-k8s-version-000272) </interface>
I0723 15:10:03.262879 61145 main.go:141] libmachine: (old-k8s-version-000272) <serial type='pty'>
I0723 15:10:03.262889 61145 main.go:141] libmachine: (old-k8s-version-000272) <target port='0'/>
I0723 15:10:03.262899 61145 main.go:141] libmachine: (old-k8s-version-000272) </serial>
I0723 15:10:03.262906 61145 main.go:141] libmachine: (old-k8s-version-000272) <console type='pty'>
I0723 15:10:03.262916 61145 main.go:141] libmachine: (old-k8s-version-000272) <target type='serial' port='0'/>
I0723 15:10:03.262927 61145 main.go:141] libmachine: (old-k8s-version-000272) </console>
I0723 15:10:03.262938 61145 main.go:141] libmachine: (old-k8s-version-000272) <rng model='virtio'>
I0723 15:10:03.262950 61145 main.go:141] libmachine: (old-k8s-version-000272) <backend model='random'>/dev/random</backend>
I0723 15:10:03.262959 61145 main.go:141] libmachine: (old-k8s-version-000272) </rng>
I0723 15:10:03.262966 61145 main.go:141] libmachine: (old-k8s-version-000272)
I0723 15:10:03.262975 61145 main.go:141] libmachine: (old-k8s-version-000272)
I0723 15:10:03.262983 61145 main.go:141] libmachine: (old-k8s-version-000272) </devices>
I0723 15:10:03.262999 61145 main.go:141] libmachine: (old-k8s-version-000272) </domain>
I0723 15:10:03.263014 61145 main.go:141] libmachine: (old-k8s-version-000272)
I0723 15:10:03.269951 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:e8:a4:22 in network default
I0723 15:10:03.270663 61145 main.go:141] libmachine: (old-k8s-version-000272) Ensuring networks are active...
I0723 15:10:03.270689 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:03.271417 61145 main.go:141] libmachine: (old-k8s-version-000272) Ensuring network default is active
I0723 15:10:03.271684 61145 main.go:141] libmachine: (old-k8s-version-000272) Ensuring network mk-old-k8s-version-000272 is active
I0723 15:10:03.272170 61145 main.go:141] libmachine: (old-k8s-version-000272) Getting domain xml...
I0723 15:10:03.272783 61145 main.go:141] libmachine: (old-k8s-version-000272) Creating domain...
I0723 15:10:04.497745 61145 main.go:141] libmachine: (old-k8s-version-000272) Waiting to get IP...
I0723 15:10:04.498732 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:04.499197 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:04.499231 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:04.499181 61404 retry.go:31] will retry after 256.08014ms: waiting for machine to come up
I0723 15:10:04.756533 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:04.756979 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:04.757005 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:04.756933 61404 retry.go:31] will retry after 259.117958ms: waiting for machine to come up
I0723 15:10:05.017188 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:05.017824 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:05.017853 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:05.017782 61404 retry.go:31] will retry after 452.717626ms: waiting for machine to come up
I0723 15:10:05.472420 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:05.472864 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:05.472892 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:05.472825 61404 retry.go:31] will retry after 460.109308ms: waiting for machine to come up
I0723 15:10:05.934510 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:05.935028 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:05.935061 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:05.934997 61404 retry.go:31] will retry after 638.200817ms: waiting for machine to come up
I0723 15:10:06.574714 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:06.575175 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:06.575206 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:06.575110 61404 retry.go:31] will retry after 857.51075ms: waiting for machine to come up
I0723 15:10:07.434309 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:07.434955 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:07.434989 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:07.434891 61404 retry.go:31] will retry after 797.566909ms: waiting for machine to come up
I0723 15:10:08.234282 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:08.234890 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:08.234918 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:08.234836 61404 retry.go:31] will retry after 1.192104413s: waiting for machine to come up
I0723 15:10:09.429296 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:09.429897 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:09.429925 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:09.429862 61404 retry.go:31] will retry after 1.377691711s: waiting for machine to come up
I0723 15:10:10.808880 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:10.809409 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:10.809438 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:10.809355 61404 retry.go:31] will retry after 1.663742638s: waiting for machine to come up
I0723 15:10:12.475323 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:12.475941 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:12.475972 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:12.475894 61404 retry.go:31] will retry after 2.728936234s: waiting for machine to come up
I0723 15:10:15.207906 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:15.208408 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:15.208436 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:15.208375 61404 retry.go:31] will retry after 3.35743868s: waiting for machine to come up
I0723 15:10:18.568024 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:18.568583 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:18.568623 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:18.568523 61404 retry.go:31] will retry after 2.740618439s: waiting for machine to come up
I0723 15:10:21.310777 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:21.311458 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find current IP address of domain old-k8s-version-000272 in network mk-old-k8s-version-000272
I0723 15:10:21.311480 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | I0723 15:10:21.311438 61404 retry.go:31] will retry after 4.881514397s: waiting for machine to come up
I0723 15:10:26.195396 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.195936 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has current primary IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.195963 61145 main.go:141] libmachine: (old-k8s-version-000272) Found IP for machine: 192.168.50.51
I0723 15:10:26.195971 61145 main.go:141] libmachine: (old-k8s-version-000272) Reserving static IP address...
I0723 15:10:26.196363 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-000272", mac: "52:54:00:90:92:e1", ip: "192.168.50.51"} in network mk-old-k8s-version-000272
I0723 15:10:26.270081 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Getting to WaitForSSH function...
I0723 15:10:26.270115 61145 main.go:141] libmachine: (old-k8s-version-000272) Reserved static IP address: 192.168.50.51
I0723 15:10:26.270134 61145 main.go:141] libmachine: (old-k8s-version-000272) Waiting for SSH to be available...
I0723 15:10:26.273446 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.273954 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:minikube Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.273991 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.274154 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Using SSH client type: external
I0723 15:10:26.274178 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Using SSH private key: /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa (-rw-------)
I0723 15:10:26.274226 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.50.51 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa -p 22] /usr/bin/ssh <nil>}
I0723 15:10:26.274244 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | About to run SSH command:
I0723 15:10:26.274261 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | exit 0
I0723 15:10:26.402609 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | SSH cmd err, output: <nil>:
I0723 15:10:26.402870 61145 main.go:141] libmachine: (old-k8s-version-000272) KVM machine creation complete!
I0723 15:10:26.403223 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetConfigRaw
I0723 15:10:26.403790 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:26.404026 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:26.404208 61145 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0723 15:10:26.404225 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetState
I0723 15:10:26.405808 61145 main.go:141] libmachine: Detecting operating system of created instance...
I0723 15:10:26.405824 61145 main.go:141] libmachine: Waiting for SSH to be available...
I0723 15:10:26.405831 61145 main.go:141] libmachine: Getting to WaitForSSH function...
I0723 15:10:26.405840 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:26.410114 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.410726 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.410755 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.410958 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:26.411117 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.411273 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.411422 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:26.411589 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:26.411857 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:26.411874 61145 main.go:141] libmachine: About to run SSH command:
exit 0
I0723 15:10:26.525605 61145 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0723 15:10:26.525628 61145 main.go:141] libmachine: Detecting the provisioner...
I0723 15:10:26.525636 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:26.528597 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.529028 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.529064 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.529198 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:26.529436 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.529634 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.529827 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:26.530013 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:26.530178 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:26.530193 61145 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0723 15:10:26.651517 61145 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0723 15:10:26.651621 61145 main.go:141] libmachine: found compatible host: buildroot
I0723 15:10:26.651643 61145 main.go:141] libmachine: Provisioning with buildroot...
I0723 15:10:26.651658 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetMachineName
I0723 15:10:26.651922 61145 buildroot.go:166] provisioning hostname "old-k8s-version-000272"
I0723 15:10:26.651956 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetMachineName
I0723 15:10:26.652166 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:26.655077 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.655500 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.655525 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.655634 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:26.655833 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.656020 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.656183 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:26.656346 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:26.656557 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:26.656572 61145 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-000272 && echo "old-k8s-version-000272" | sudo tee /etc/hostname
I0723 15:10:26.791952 61145 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-000272
I0723 15:10:26.791987 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:26.795425 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.795834 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.795861 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.796085 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:26.796283 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.796443 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:26.796605 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:26.796827 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:26.797074 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:26.797103 61145 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-000272' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-000272/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-000272' | sudo tee -a /etc/hosts;
fi
fi
I0723 15:10:26.928991 61145 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0723 15:10:26.929030 61145 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19319-11303/.minikube CaCertPath:/home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19319-11303/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19319-11303/.minikube}
I0723 15:10:26.929087 61145 buildroot.go:174] setting up certificates
I0723 15:10:26.929101 61145 provision.go:84] configureAuth start
I0723 15:10:26.929121 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetMachineName
I0723 15:10:26.929437 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetIP
I0723 15:10:26.933166 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.933581 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.933627 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.933823 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:26.936982 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.937481 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:26.937566 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:26.937805 61145 provision.go:143] copyHostCerts
I0723 15:10:26.937866 61145 exec_runner.go:144] found /home/jenkins/minikube-integration/19319-11303/.minikube/ca.pem, removing ...
I0723 15:10:26.937881 61145 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19319-11303/.minikube/ca.pem
I0723 15:10:26.937953 61145 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19319-11303/.minikube/ca.pem (1078 bytes)
I0723 15:10:26.938100 61145 exec_runner.go:144] found /home/jenkins/minikube-integration/19319-11303/.minikube/cert.pem, removing ...
I0723 15:10:26.938110 61145 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19319-11303/.minikube/cert.pem
I0723 15:10:26.938145 61145 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19319-11303/.minikube/cert.pem (1123 bytes)
I0723 15:10:26.938253 61145 exec_runner.go:144] found /home/jenkins/minikube-integration/19319-11303/.minikube/key.pem, removing ...
I0723 15:10:26.938265 61145 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19319-11303/.minikube/key.pem
I0723 15:10:26.938296 61145 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19319-11303/.minikube/key.pem (1675 bytes)
I0723 15:10:26.938401 61145 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19319-11303/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-000272 san=[127.0.0.1 192.168.50.51 localhost minikube old-k8s-version-000272]
I0723 15:10:27.049008 61145 provision.go:177] copyRemoteCerts
I0723 15:10:27.049075 61145 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0723 15:10:27.049098 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.052233 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.052659 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.052689 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.052874 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.053077 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.053268 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.053405 61145 sshutil.go:53] new ssh client: &{IP:192.168.50.51 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa Username:docker}
I0723 15:10:27.147760 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0723 15:10:27.172799 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0723 15:10:27.196252 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0723 15:10:27.224873 61145 provision.go:87] duration metric: took 295.754971ms to configureAuth
I0723 15:10:27.224905 61145 buildroot.go:189] setting minikube options for container-runtime
I0723 15:10:27.225103 61145 config.go:182] Loaded profile config "old-k8s-version-000272": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0723 15:10:27.225201 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.228098 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.228533 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.228563 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.228736 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.228972 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.229167 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.229338 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.229504 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:27.229669 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:27.229683 61145 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0723 15:10:27.503248 61145 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0723 15:10:27.503301 61145 main.go:141] libmachine: Checking connection to Docker...
I0723 15:10:27.503311 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetURL
I0723 15:10:27.504599 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | Using libvirt version 6000000
I0723 15:10:27.506786 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.507139 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.507165 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.507324 61145 main.go:141] libmachine: Docker is up and running!
I0723 15:10:27.507339 61145 main.go:141] libmachine: Reticulating splines...
I0723 15:10:27.507345 61145 client.go:171] duration metric: took 24.6115202s to LocalClient.Create
I0723 15:10:27.507367 61145 start.go:167] duration metric: took 24.611614283s to libmachine.API.Create "old-k8s-version-000272"
I0723 15:10:27.507377 61145 start.go:293] postStartSetup for "old-k8s-version-000272" (driver="kvm2")
I0723 15:10:27.507387 61145 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0723 15:10:27.507401 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:27.507614 61145 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0723 15:10:27.507640 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.509822 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.510110 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.510157 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.510240 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.510433 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.510593 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.510717 61145 sshutil.go:53] new ssh client: &{IP:192.168.50.51 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa Username:docker}
I0723 15:10:27.596741 61145 ssh_runner.go:195] Run: cat /etc/os-release
I0723 15:10:27.600947 61145 info.go:137] Remote host: Buildroot 2023.02.9
I0723 15:10:27.600971 61145 filesync.go:126] Scanning /home/jenkins/minikube-integration/19319-11303/.minikube/addons for local assets ...
I0723 15:10:27.601029 61145 filesync.go:126] Scanning /home/jenkins/minikube-integration/19319-11303/.minikube/files for local assets ...
I0723 15:10:27.601108 61145 filesync.go:149] local asset: /home/jenkins/minikube-integration/19319-11303/.minikube/files/etc/ssl/certs/185032.pem -> 185032.pem in /etc/ssl/certs
I0723 15:10:27.601197 61145 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0723 15:10:27.610724 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/files/etc/ssl/certs/185032.pem --> /etc/ssl/certs/185032.pem (1708 bytes)
I0723 15:10:27.633680 61145 start.go:296] duration metric: took 126.289247ms for postStartSetup
I0723 15:10:27.633739 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetConfigRaw
I0723 15:10:27.634331 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetIP
I0723 15:10:27.637329 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.637748 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.637791 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.637990 61145 profile.go:143] Saving config to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/config.json ...
I0723 15:10:27.638180 61145 start.go:128] duration metric: took 24.763180023s to createHost
I0723 15:10:27.638204 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.640376 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.640679 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.640705 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.640852 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.641041 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.641186 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.641320 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.641497 61145 main.go:141] libmachine: Using SSH client type: native
I0723 15:10:27.641779 61145 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.50.51 22 <nil> <nil>}
I0723 15:10:27.641797 61145 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0723 15:10:27.759242 61145 main.go:141] libmachine: SSH cmd err, output: <nil>: 1721747427.719404535
I0723 15:10:27.759267 61145 fix.go:216] guest clock: 1721747427.719404535
I0723 15:10:27.759278 61145 fix.go:229] Guest: 2024-07-23 15:10:27.719404535 +0000 UTC Remote: 2024-07-23 15:10:27.638192314 +0000 UTC m=+47.452928765 (delta=81.212221ms)
I0723 15:10:27.759308 61145 fix.go:200] guest clock delta is within tolerance: 81.212221ms
I0723 15:10:27.759314 61145 start.go:83] releasing machines lock for "old-k8s-version-000272", held for 24.884500576s
I0723 15:10:27.759334 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:27.759588 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetIP
I0723 15:10:27.762659 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.763182 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.763221 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.763347 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:27.763888 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:27.764103 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .DriverName
I0723 15:10:27.764257 61145 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0723 15:10:27.764323 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.764367 61145 ssh_runner.go:195] Run: cat /version.json
I0723 15:10:27.764405 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHHostname
I0723 15:10:27.767981 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.768308 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.768337 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.768356 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.768509 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.768705 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.768917 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:27.768956 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:27.769025 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHPort
I0723 15:10:27.769061 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.769186 61145 sshutil.go:53] new ssh client: &{IP:192.168.50.51 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa Username:docker}
I0723 15:10:27.769672 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHKeyPath
I0723 15:10:27.769822 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetSSHUsername
I0723 15:10:27.769964 61145 sshutil.go:53] new ssh client: &{IP:192.168.50.51 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19319-11303/.minikube/machines/old-k8s-version-000272/id_rsa Username:docker}
I0723 15:10:27.858950 61145 ssh_runner.go:195] Run: systemctl --version
I0723 15:10:27.899027 61145 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0723 15:10:28.062646 61145 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0723 15:10:28.069186 61145 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0723 15:10:28.069257 61145 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0723 15:10:28.084346 61145 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0723 15:10:28.084377 61145 start.go:495] detecting cgroup driver to use...
I0723 15:10:28.084451 61145 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0723 15:10:28.099803 61145 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0723 15:10:28.113201 61145 docker.go:217] disabling cri-docker service (if available) ...
I0723 15:10:28.113253 61145 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0723 15:10:28.126629 61145 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0723 15:10:28.140106 61145 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0723 15:10:28.259587 61145 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0723 15:10:28.397192 61145 docker.go:233] disabling docker service ...
I0723 15:10:28.397266 61145 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0723 15:10:28.411734 61145 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0723 15:10:28.424498 61145 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0723 15:10:28.565776 61145 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0723 15:10:28.676941 61145 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0723 15:10:28.689914 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0723 15:10:28.709235 61145 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0723 15:10:28.709301 61145 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0723 15:10:28.719250 61145 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0723 15:10:28.719316 61145 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0723 15:10:28.729925 61145 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0723 15:10:28.739817 61145 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0723 15:10:28.750020 61145 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0723 15:10:28.760050 61145 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0723 15:10:28.769140 61145 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0723 15:10:28.769204 61145 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0723 15:10:28.780710 61145 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0723 15:10:28.789727 61145 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0723 15:10:28.907710 61145 ssh_runner.go:195] Run: sudo systemctl restart crio
I0723 15:10:29.069526 61145 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0723 15:10:29.069594 61145 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0723 15:10:29.075194 61145 start.go:563] Will wait 60s for crictl version
I0723 15:10:29.075256 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:29.079759 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0723 15:10:29.117624 61145 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0723 15:10:29.117713 61145 ssh_runner.go:195] Run: crio --version
I0723 15:10:29.146900 61145 ssh_runner.go:195] Run: crio --version
I0723 15:10:29.181963 61145 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0723 15:10:29.183268 61145 main.go:141] libmachine: (old-k8s-version-000272) Calling .GetIP
I0723 15:10:29.186302 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:29.186876 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:90:92:e1", ip: ""} in network mk-old-k8s-version-000272: {Iface:virbr4 ExpiryTime:2024-07-23 16:10:17 +0000 UTC Type:0 Mac:52:54:00:90:92:e1 Iaid: IPaddr:192.168.50.51 Prefix:24 Hostname:old-k8s-version-000272 Clientid:01:52:54:00:90:92:e1}
I0723 15:10:29.186907 61145 main.go:141] libmachine: (old-k8s-version-000272) DBG | domain old-k8s-version-000272 has defined IP address 192.168.50.51 and MAC address 52:54:00:90:92:e1 in network mk-old-k8s-version-000272
I0723 15:10:29.187185 61145 ssh_runner.go:195] Run: grep 192.168.50.1 host.minikube.internal$ /etc/hosts
I0723 15:10:29.192072 61145 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.50.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0723 15:10:29.204717 61145 kubeadm.go:883] updating cluster {Name:old-k8s-version-000272 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721687125-19319@sha256:8e08c9232f6be53a68fac8937bc5a03a99e38d6a1ff7b6ea4048c989041004ae Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-000272 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.51 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0723 15:10:29.204827 61145 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0723 15:10:29.204886 61145 ssh_runner.go:195] Run: sudo crictl images --output json
I0723 15:10:29.238866 61145 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0723 15:10:29.238965 61145 ssh_runner.go:195] Run: which lz4
I0723 15:10:29.243219 61145 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0723 15:10:29.247311 61145 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0723 15:10:29.247351 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0723 15:10:30.656817 61145 crio.go:462] duration metric: took 1.413630732s to copy over tarball
I0723 15:10:30.656906 61145 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0723 15:10:33.138312 61145 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.481376729s)
I0723 15:10:33.138348 61145 crio.go:469] duration metric: took 2.481502405s to extract the tarball
I0723 15:10:33.138356 61145 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0723 15:10:33.179771 61145 ssh_runner.go:195] Run: sudo crictl images --output json
I0723 15:10:33.224053 61145 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0723 15:10:33.224078 61145 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0723 15:10:33.224136 61145 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0723 15:10:33.224151 61145 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0723 15:10:33.224168 61145 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0723 15:10:33.224198 61145 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0723 15:10:33.224221 61145 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0723 15:10:33.224237 61145 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0723 15:10:33.224202 61145 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0723 15:10:33.224393 61145 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0723 15:10:33.225600 61145 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0723 15:10:33.225606 61145 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0723 15:10:33.225676 61145 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0723 15:10:33.225683 61145 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0723 15:10:33.225672 61145 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0723 15:10:33.225752 61145 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0723 15:10:33.225764 61145 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0723 15:10:33.225979 61145 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0723 15:10:33.484728 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0723 15:10:33.494363 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0723 15:10:33.506515 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0723 15:10:33.515497 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0723 15:10:33.526221 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0723 15:10:33.530787 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0723 15:10:33.531804 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0723 15:10:33.560732 61145 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0723 15:10:33.560778 61145 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0723 15:10:33.560812 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.597336 61145 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0723 15:10:33.597378 61145 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0723 15:10:33.597426 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.656196 61145 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0723 15:10:33.656238 61145 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0723 15:10:33.656290 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.656303 61145 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0723 15:10:33.656351 61145 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0723 15:10:33.656408 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.671454 61145 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0723 15:10:33.671487 61145 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0723 15:10:33.671491 61145 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0723 15:10:33.671499 61145 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0723 15:10:33.671517 61145 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0723 15:10:33.671517 61145 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0723 15:10:33.671545 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.671550 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.671551 61145 ssh_runner.go:195] Run: which crictl
I0723 15:10:33.671568 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0723 15:10:33.671607 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0723 15:10:33.671630 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0723 15:10:33.671654 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0723 15:10:33.749652 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0723 15:10:33.749723 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0723 15:10:33.749735 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0723 15:10:33.749787 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0723 15:10:33.749801 61145 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0723 15:10:33.755999 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0723 15:10:33.756075 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0723 15:10:33.826569 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0723 15:10:33.826617 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0723 15:10:33.826658 61145 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0723 15:10:34.201757 61145 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0723 15:10:34.342065 61145 cache_images.go:92] duration metric: took 1.117972044s to LoadCachedImages
W0723 15:10:34.342156 61145 out.go:239] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19319-11303/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0723 15:10:34.342170 61145 kubeadm.go:934] updating node { 192.168.50.51 8443 v1.20.0 crio true true} ...
I0723 15:10:34.342302 61145 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-000272 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.50.51
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-000272 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0723 15:10:34.342403 61145 ssh_runner.go:195] Run: crio config
I0723 15:10:34.395196 61145 cni.go:84] Creating CNI manager for ""
I0723 15:10:34.395218 61145 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0723 15:10:34.395230 61145 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0723 15:10:34.395247 61145 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.50.51 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-000272 NodeName:old-k8s-version-000272 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.50.51"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.50.51 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0723 15:10:34.395369 61145 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.50.51
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-000272"
kubeletExtraArgs:
node-ip: 192.168.50.51
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.50.51"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0723 15:10:34.395427 61145 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0723 15:10:34.405241 61145 binaries.go:44] Found k8s binaries, skipping transfer
I0723 15:10:34.405319 61145 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0723 15:10:34.415011 61145 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0723 15:10:34.430524 61145 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0723 15:10:34.445887 61145 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0723 15:10:34.461971 61145 ssh_runner.go:195] Run: grep 192.168.50.51 control-plane.minikube.internal$ /etc/hosts
I0723 15:10:34.465470 61145 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.50.51 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0723 15:10:34.476915 61145 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0723 15:10:34.609342 61145 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0723 15:10:34.625056 61145 certs.go:68] Setting up /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272 for IP: 192.168.50.51
I0723 15:10:34.625087 61145 certs.go:194] generating shared ca certs ...
I0723 15:10:34.625104 61145 certs.go:226] acquiring lock for ca certs: {Name:mkc9951d8e001787fba4648f53fcd0a765dde2e1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.625294 61145 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19319-11303/.minikube/ca.key
I0723 15:10:34.625375 61145 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19319-11303/.minikube/proxy-client-ca.key
I0723 15:10:34.625392 61145 certs.go:256] generating profile certs ...
I0723 15:10:34.625459 61145 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.key
I0723 15:10:34.625481 61145 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.crt with IP's: []
I0723 15:10:34.699958 61145 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.crt ...
I0723 15:10:34.699994 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.crt: {Name:mka8c750715c67ebef618d5588712c67401d26a3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.700189 61145 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.key ...
I0723 15:10:34.700206 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/client.key: {Name:mk682025ce1b19888be5c12fcc2fd30240d2ff1d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.700333 61145 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key.2c7d9ab3
I0723 15:10:34.700358 61145 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt.2c7d9ab3 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.50.51]
I0723 15:10:34.812154 61145 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt.2c7d9ab3 ...
I0723 15:10:34.812182 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt.2c7d9ab3: {Name:mk1254cb017d09383da5ef370629d42bdf6d88ef Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.812368 61145 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key.2c7d9ab3 ...
I0723 15:10:34.812388 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key.2c7d9ab3: {Name:mkfcb32affd2bac243be7acf32499df9cb786f1d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.812479 61145 certs.go:381] copying /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt.2c7d9ab3 -> /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt
I0723 15:10:34.812563 61145 certs.go:385] copying /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key.2c7d9ab3 -> /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key
I0723 15:10:34.812624 61145 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.key
I0723 15:10:34.812663 61145 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.crt with IP's: []
I0723 15:10:34.968992 61145 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.crt ...
I0723 15:10:34.969019 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.crt: {Name:mk7c499cf2d149f4efc9d085464f7cce66885ae2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.969189 61145 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.key ...
I0723 15:10:34.969208 61145 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.key: {Name:mkb9b32fbe9e0f68e7d471fbb5c262566e3a92dd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0723 15:10:34.969450 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/18503.pem (1338 bytes)
W0723 15:10:34.969492 61145 certs.go:480] ignoring /home/jenkins/minikube-integration/19319-11303/.minikube/certs/18503_empty.pem, impossibly tiny 0 bytes
I0723 15:10:34.969500 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca-key.pem (1679 bytes)
I0723 15:10:34.969540 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/ca.pem (1078 bytes)
I0723 15:10:34.969572 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/cert.pem (1123 bytes)
I0723 15:10:34.969605 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/certs/key.pem (1675 bytes)
I0723 15:10:34.969662 61145 certs.go:484] found cert: /home/jenkins/minikube-integration/19319-11303/.minikube/files/etc/ssl/certs/185032.pem (1708 bytes)
I0723 15:10:34.970273 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0723 15:10:34.994940 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0723 15:10:35.016898 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0723 15:10:35.038793 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0723 15:10:35.060364 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0723 15:10:35.082065 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0723 15:10:35.104784 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0723 15:10:35.127683 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/profiles/old-k8s-version-000272/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0723 15:10:35.150417 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0723 15:10:35.173379 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/certs/18503.pem --> /usr/share/ca-certificates/18503.pem (1338 bytes)
I0723 15:10:35.196144 61145 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19319-11303/.minikube/files/etc/ssl/certs/185032.pem --> /usr/share/ca-certificates/185032.pem (1708 bytes)
I0723 15:10:35.219013 61145 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0723 15:10:35.234575 61145 ssh_runner.go:195] Run: openssl version
I0723 15:10:35.239886 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/185032.pem && ln -fs /usr/share/ca-certificates/185032.pem /etc/ssl/certs/185032.pem"
I0723 15:10:35.249998 61145 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/185032.pem
I0723 15:10:35.254127 61145 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jul 23 14:09 /usr/share/ca-certificates/185032.pem
I0723 15:10:35.254189 61145 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/185032.pem
I0723 15:10:35.259823 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/185032.pem /etc/ssl/certs/3ec20f2e.0"
I0723 15:10:35.270300 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0723 15:10:35.280921 61145 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0723 15:10:35.285249 61145 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jul 23 13:57 /usr/share/ca-certificates/minikubeCA.pem
I0723 15:10:35.285309 61145 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0723 15:10:35.290681 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0723 15:10:35.301013 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/18503.pem && ln -fs /usr/share/ca-certificates/18503.pem /etc/ssl/certs/18503.pem"
I0723 15:10:35.311662 61145 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/18503.pem
I0723 15:10:35.316014 61145 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jul 23 14:09 /usr/share/ca-certificates/18503.pem
I0723 15:10:35.316067 61145 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/18503.pem
I0723 15:10:35.321697 61145 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/18503.pem /etc/ssl/certs/51391683.0"
I0723 15:10:35.332062 61145 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0723 15:10:35.335824 61145 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0723 15:10:35.335879 61145 kubeadm.go:392] StartCluster: {Name:old-k8s-version-000272 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721687125-19319@sha256:8e08c9232f6be53a68fac8937bc5a03a99e38d6a1ff7b6ea4048c989041004ae Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-000272 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.51 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0723 15:10:35.335980 61145 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0723 15:10:35.336031 61145 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0723 15:10:35.372538 61145 cri.go:89] found id: ""
I0723 15:10:35.372623 61145 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0723 15:10:35.382303 61145 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0723 15:10:35.391547 61145 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0723 15:10:35.400992 61145 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0723 15:10:35.401015 61145 kubeadm.go:157] found existing configuration files:
I0723 15:10:35.401061 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0723 15:10:35.412310 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0723 15:10:35.412371 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0723 15:10:35.421650 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0723 15:10:35.438246 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0723 15:10:35.438328 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0723 15:10:35.449289 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0723 15:10:35.459793 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0723 15:10:35.459850 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0723 15:10:35.471471 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0723 15:10:35.483265 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0723 15:10:35.483317 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0723 15:10:35.497890 61145 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0723 15:10:35.771742 61145 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0723 15:12:33.782566 61145 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0723 15:12:33.782779 61145 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0723 15:12:33.783898 61145 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0723 15:12:33.783999 61145 kubeadm.go:310] [preflight] Running pre-flight checks
I0723 15:12:33.784179 61145 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0723 15:12:33.784396 61145 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0723 15:12:33.784673 61145 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0723 15:12:33.784930 61145 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0723 15:12:33.786874 61145 out.go:204] - Generating certificates and keys ...
I0723 15:12:33.786970 61145 kubeadm.go:310] [certs] Using existing ca certificate authority
I0723 15:12:33.787050 61145 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0723 15:12:33.787109 61145 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0723 15:12:33.787182 61145 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0723 15:12:33.787256 61145 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0723 15:12:33.787305 61145 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0723 15:12:33.787371 61145 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0723 15:12:33.787501 61145 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
I0723 15:12:33.787599 61145 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0723 15:12:33.787743 61145 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
I0723 15:12:33.787834 61145 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0723 15:12:33.787912 61145 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0723 15:12:33.787950 61145 kubeadm.go:310] [certs] Generating "sa" key and public key
I0723 15:12:33.787998 61145 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0723 15:12:33.788037 61145 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0723 15:12:33.788082 61145 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0723 15:12:33.788131 61145 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0723 15:12:33.788173 61145 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0723 15:12:33.788255 61145 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0723 15:12:33.788331 61145 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0723 15:12:33.788362 61145 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0723 15:12:33.788414 61145 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0723 15:12:33.790124 61145 out.go:204] - Booting up control plane ...
I0723 15:12:33.790213 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0723 15:12:33.790279 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0723 15:12:33.790342 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0723 15:12:33.790425 61145 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0723 15:12:33.790545 61145 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0723 15:12:33.790591 61145 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0723 15:12:33.790648 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:12:33.790797 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:12:33.790851 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:12:33.791065 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:12:33.791131 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:12:33.791285 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:12:33.791350 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:12:33.791538 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:12:33.791620 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:12:33.791832 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:12:33.791843 61145 kubeadm.go:310]
I0723 15:12:33.791896 61145 kubeadm.go:310] Unfortunately, an error has occurred:
I0723 15:12:33.791982 61145 kubeadm.go:310] timed out waiting for the condition
I0723 15:12:33.791999 61145 kubeadm.go:310]
I0723 15:12:33.792051 61145 kubeadm.go:310] This error is likely caused by:
I0723 15:12:33.792090 61145 kubeadm.go:310] - The kubelet is not running
I0723 15:12:33.792219 61145 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0723 15:12:33.792231 61145 kubeadm.go:310]
I0723 15:12:33.792360 61145 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0723 15:12:33.792397 61145 kubeadm.go:310] - 'systemctl status kubelet'
I0723 15:12:33.792425 61145 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0723 15:12:33.792431 61145 kubeadm.go:310]
I0723 15:12:33.792585 61145 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0723 15:12:33.792678 61145 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0723 15:12:33.792694 61145 kubeadm.go:310]
I0723 15:12:33.792820 61145 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0723 15:12:33.792940 61145 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0723 15:12:33.793036 61145 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0723 15:12:33.793134 61145 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0723 15:12:33.793187 61145 kubeadm.go:310]
W0723 15:12:33.793258 61145 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-000272] and IPs [192.168.50.51 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0723 15:12:33.793311 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0723 15:12:34.284392 61145 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0723 15:12:34.298350 61145 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0723 15:12:34.307570 61145 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0723 15:12:34.307593 61145 kubeadm.go:157] found existing configuration files:
I0723 15:12:34.307634 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0723 15:12:34.316580 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0723 15:12:34.316649 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0723 15:12:34.325775 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0723 15:12:34.334638 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0723 15:12:34.334700 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0723 15:12:34.343659 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0723 15:12:34.351993 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0723 15:12:34.352051 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0723 15:12:34.360837 61145 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0723 15:12:34.369182 61145 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0723 15:12:34.369235 61145 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0723 15:12:34.377889 61145 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0723 15:12:34.443277 61145 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0723 15:12:34.443349 61145 kubeadm.go:310] [preflight] Running pre-flight checks
I0723 15:12:34.585423 61145 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0723 15:12:34.585523 61145 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0723 15:12:34.585614 61145 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0723 15:12:34.758272 61145 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0723 15:12:34.760811 61145 out.go:204] - Generating certificates and keys ...
I0723 15:12:34.760915 61145 kubeadm.go:310] [certs] Using existing ca certificate authority
I0723 15:12:34.761002 61145 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0723 15:12:34.761113 61145 kubeadm.go:310] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0723 15:12:34.761205 61145 kubeadm.go:310] [certs] Using existing front-proxy-ca certificate authority
I0723 15:12:34.761316 61145 kubeadm.go:310] [certs] Using existing front-proxy-client certificate and key on disk
I0723 15:12:34.761391 61145 kubeadm.go:310] [certs] Using existing etcd/ca certificate authority
I0723 15:12:34.761447 61145 kubeadm.go:310] [certs] Using existing etcd/server certificate and key on disk
I0723 15:12:34.761503 61145 kubeadm.go:310] [certs] Using existing etcd/peer certificate and key on disk
I0723 15:12:34.761596 61145 kubeadm.go:310] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0723 15:12:34.761718 61145 kubeadm.go:310] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0723 15:12:34.761784 61145 kubeadm.go:310] [certs] Using the existing "sa" key
I0723 15:12:34.761864 61145 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0723 15:12:34.878098 61145 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0723 15:12:34.967030 61145 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0723 15:12:35.295067 61145 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0723 15:12:35.413898 61145 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0723 15:12:35.429452 61145 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0723 15:12:35.430651 61145 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0723 15:12:35.430771 61145 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0723 15:12:35.569029 61145 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0723 15:12:35.570664 61145 out.go:204] - Booting up control plane ...
I0723 15:12:35.570796 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0723 15:12:35.576642 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0723 15:12:35.577955 61145 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0723 15:12:35.579387 61145 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0723 15:12:35.582369 61145 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0723 15:13:15.584418 61145 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0723 15:13:15.584899 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:13:15.585145 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:13:20.585685 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:13:20.585931 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:13:30.586788 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:13:30.587080 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:13:50.588334 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:13:50.588540 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:14:30.587817 61145 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0723 15:14:30.588089 61145 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0723 15:14:30.588115 61145 kubeadm.go:310]
I0723 15:14:30.588169 61145 kubeadm.go:310] Unfortunately, an error has occurred:
I0723 15:14:30.588227 61145 kubeadm.go:310] timed out waiting for the condition
I0723 15:14:30.588237 61145 kubeadm.go:310]
I0723 15:14:30.588278 61145 kubeadm.go:310] This error is likely caused by:
I0723 15:14:30.588331 61145 kubeadm.go:310] - The kubelet is not running
I0723 15:14:30.588483 61145 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0723 15:14:30.588496 61145 kubeadm.go:310]
I0723 15:14:30.588644 61145 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0723 15:14:30.588705 61145 kubeadm.go:310] - 'systemctl status kubelet'
I0723 15:14:30.588750 61145 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0723 15:14:30.588761 61145 kubeadm.go:310]
I0723 15:14:30.588891 61145 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0723 15:14:30.589002 61145 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0723 15:14:30.589015 61145 kubeadm.go:310]
I0723 15:14:30.589170 61145 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0723 15:14:30.589428 61145 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0723 15:14:30.589583 61145 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0723 15:14:30.589691 61145 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0723 15:14:30.589702 61145 kubeadm.go:310]
I0723 15:14:30.590661 61145 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0723 15:14:30.590791 61145 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0723 15:14:30.590890 61145 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0723 15:14:30.590961 61145 kubeadm.go:394] duration metric: took 3m55.255086134s to StartCluster
I0723 15:14:30.591024 61145 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0723 15:14:30.591087 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0723 15:14:30.640630 61145 cri.go:89] found id: ""
I0723 15:14:30.640658 61145 logs.go:276] 0 containers: []
W0723 15:14:30.640669 61145 logs.go:278] No container was found matching "kube-apiserver"
I0723 15:14:30.640676 61145 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0723 15:14:30.640732 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0723 15:14:30.678920 61145 cri.go:89] found id: ""
I0723 15:14:30.678946 61145 logs.go:276] 0 containers: []
W0723 15:14:30.678954 61145 logs.go:278] No container was found matching "etcd"
I0723 15:14:30.678962 61145 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0723 15:14:30.679023 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0723 15:14:30.717609 61145 cri.go:89] found id: ""
I0723 15:14:30.717633 61145 logs.go:276] 0 containers: []
W0723 15:14:30.717642 61145 logs.go:278] No container was found matching "coredns"
I0723 15:14:30.717649 61145 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0723 15:14:30.717700 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0723 15:14:30.775955 61145 cri.go:89] found id: ""
I0723 15:14:30.775986 61145 logs.go:276] 0 containers: []
W0723 15:14:30.775995 61145 logs.go:278] No container was found matching "kube-scheduler"
I0723 15:14:30.776003 61145 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0723 15:14:30.776069 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0723 15:14:30.810116 61145 cri.go:89] found id: ""
I0723 15:14:30.810144 61145 logs.go:276] 0 containers: []
W0723 15:14:30.810155 61145 logs.go:278] No container was found matching "kube-proxy"
I0723 15:14:30.810163 61145 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0723 15:14:30.810224 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0723 15:14:30.844174 61145 cri.go:89] found id: ""
I0723 15:14:30.844203 61145 logs.go:276] 0 containers: []
W0723 15:14:30.844214 61145 logs.go:278] No container was found matching "kube-controller-manager"
I0723 15:14:30.844222 61145 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0723 15:14:30.844284 61145 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0723 15:14:30.880638 61145 cri.go:89] found id: ""
I0723 15:14:30.880671 61145 logs.go:276] 0 containers: []
W0723 15:14:30.880681 61145 logs.go:278] No container was found matching "kindnet"
I0723 15:14:30.880693 61145 logs.go:123] Gathering logs for dmesg ...
I0723 15:14:30.880709 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0723 15:14:30.895113 61145 logs.go:123] Gathering logs for describe nodes ...
I0723 15:14:30.895140 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0723 15:14:31.031289 61145 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0723 15:14:31.031311 61145 logs.go:123] Gathering logs for CRI-O ...
I0723 15:14:31.031325 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0723 15:14:31.131316 61145 logs.go:123] Gathering logs for container status ...
I0723 15:14:31.131352 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0723 15:14:31.181186 61145 logs.go:123] Gathering logs for kubelet ...
I0723 15:14:31.181215 61145 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0723 15:14:31.230540 61145 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0723 15:14:31.230594 61145 out.go:239] *
*
W0723 15:14:31.230658 61145 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0723 15:14:31.230686 61145 out.go:239] *
*
W0723 15:14:31.231504 61145 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0723 15:14:31.234028 61145 out.go:177]
W0723 15:14:31.235075 61145 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0723 15:14:31.235116 61145 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0723 15:14:31.235141 61145 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0723 15:14:31.236539 61145 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-000272 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-000272 -n old-k8s-version-000272
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-000272 -n old-k8s-version-000272: exit status 6 (239.241867ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0723 15:14:31.508559 64063 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-000272" does not appear in /home/jenkins/minikube-integration/19319-11303/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-000272" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (291.34s)