=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-159022 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-159022 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (5m11.231209706s)
-- stdout --
* [old-k8s-version-159022] minikube v1.32.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=18453
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18453-10028/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18453-10028/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-159022" primary control-plane node in "old-k8s-version-159022" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0319 20:24:46.331797 55982 out.go:291] Setting OutFile to fd 1 ...
I0319 20:24:46.331931 55982 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0319 20:24:46.331944 55982 out.go:304] Setting ErrFile to fd 2...
I0319 20:24:46.331960 55982 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0319 20:24:46.332662 55982 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18453-10028/.minikube/bin
I0319 20:24:46.333546 55982 out.go:298] Setting JSON to false
I0319 20:24:46.335054 55982 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-10","uptime":7584,"bootTime":1710872302,"procs":222,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1054-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0319 20:24:46.335213 55982 start.go:139] virtualization: kvm guest
I0319 20:24:46.337616 55982 out.go:177] * [old-k8s-version-159022] minikube v1.32.0 on Ubuntu 20.04 (kvm/amd64)
I0319 20:24:46.339583 55982 out.go:177] - MINIKUBE_LOCATION=18453
I0319 20:24:46.340895 55982 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0319 20:24:46.339625 55982 notify.go:220] Checking for updates...
I0319 20:24:46.343386 55982 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18453-10028/kubeconfig
I0319 20:24:46.344952 55982 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18453-10028/.minikube
I0319 20:24:46.346373 55982 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0319 20:24:46.347682 55982 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0319 20:24:46.349542 55982 config.go:182] Loaded profile config "cert-expiration-428153": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0319 20:24:46.349683 55982 config.go:182] Loaded profile config "cert-options-346618": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0319 20:24:46.349798 55982 config.go:182] Loaded profile config "kubernetes-upgrade-853797": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.0-beta.0
I0319 20:24:46.349934 55982 driver.go:392] Setting default libvirt URI to qemu:///system
I0319 20:24:46.386278 55982 out.go:177] * Using the kvm2 driver based on user configuration
I0319 20:24:46.387877 55982 start.go:297] selected driver: kvm2
I0319 20:24:46.387892 55982 start.go:901] validating driver "kvm2" against <nil>
I0319 20:24:46.387906 55982 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0319 20:24:46.388979 55982 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0319 20:24:46.389052 55982 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/18453-10028/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0319 20:24:46.405094 55982 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.32.0
I0319 20:24:46.405146 55982 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0319 20:24:46.405417 55982 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0319 20:24:46.405498 55982 cni.go:84] Creating CNI manager for ""
I0319 20:24:46.405516 55982 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0319 20:24:46.405528 55982 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0319 20:24:46.405600 55982 start.go:340] cluster config:
{Name:old-k8s-version-159022 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-159022 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0319 20:24:46.405718 55982 iso.go:125] acquiring lock: {Name:mk757175fceba09a5d2cb7ea19c00dcf80754cf1 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0319 20:24:46.407838 55982 out.go:177] * Starting "old-k8s-version-159022" primary control-plane node in "old-k8s-version-159022" cluster
I0319 20:24:46.409431 55982 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0319 20:24:46.409470 55982 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0319 20:24:46.409482 55982 cache.go:56] Caching tarball of preloaded images
I0319 20:24:46.409583 55982 preload.go:173] Found /home/jenkins/minikube-integration/18453-10028/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0319 20:24:46.409596 55982 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0319 20:24:46.409719 55982 profile.go:142] Saving config to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/config.json ...
I0319 20:24:46.409742 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/config.json: {Name:mkb91457dccfd07540af13c7977b564d34007a78 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:24:46.409887 55982 start.go:360] acquireMachinesLock for old-k8s-version-159022: {Name:mk40947b31effb7c3f1078cbd662c574a0260f3d Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0319 20:25:20.013507 55982 start.go:364] duration metric: took 33.603586686s to acquireMachinesLock for "old-k8s-version-159022"
I0319 20:25:20.013579 55982 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-159022 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18277/minikube-v1.32.1-1710573846-18277-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-159022 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0319 20:25:20.013715 55982 start.go:125] createHost starting for "" (driver="kvm2")
I0319 20:25:20.016016 55982 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0319 20:25:20.016216 55982 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0319 20:25:20.016272 55982 main.go:141] libmachine: Launching plugin server for driver kvm2
I0319 20:25:20.032911 55982 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:44463
I0319 20:25:20.033334 55982 main.go:141] libmachine: () Calling .GetVersion
I0319 20:25:20.033953 55982 main.go:141] libmachine: Using API Version 1
I0319 20:25:20.033981 55982 main.go:141] libmachine: () Calling .SetConfigRaw
I0319 20:25:20.034279 55982 main.go:141] libmachine: () Calling .GetMachineName
I0319 20:25:20.034454 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetMachineName
I0319 20:25:20.034596 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:20.034732 55982 start.go:159] libmachine.API.Create for "old-k8s-version-159022" (driver="kvm2")
I0319 20:25:20.034761 55982 client.go:168] LocalClient.Create starting
I0319 20:25:20.034790 55982 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem
I0319 20:25:20.034822 55982 main.go:141] libmachine: Decoding PEM data...
I0319 20:25:20.034836 55982 main.go:141] libmachine: Parsing certificate...
I0319 20:25:20.034885 55982 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18453-10028/.minikube/certs/cert.pem
I0319 20:25:20.034903 55982 main.go:141] libmachine: Decoding PEM data...
I0319 20:25:20.034914 55982 main.go:141] libmachine: Parsing certificate...
I0319 20:25:20.034930 55982 main.go:141] libmachine: Running pre-create checks...
I0319 20:25:20.034939 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .PreCreateCheck
I0319 20:25:20.035232 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetConfigRaw
I0319 20:25:20.035646 55982 main.go:141] libmachine: Creating machine...
I0319 20:25:20.035664 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .Create
I0319 20:25:20.035810 55982 main.go:141] libmachine: (old-k8s-version-159022) Creating KVM machine...
I0319 20:25:20.036933 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found existing default KVM network
I0319 20:25:20.037991 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.037847 56546 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:b6:20:18} reservation:<nil>}
I0319 20:25:20.038687 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.038572 56546 network.go:211] skipping subnet 192.168.50.0/24 that is taken: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName:virbr2 IfaceIPv4:192.168.50.1 IfaceMTU:1500 IfaceMAC:52:54:00:c9:58:8d} reservation:<nil>}
I0319 20:25:20.039469 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.039386 56546 network.go:206] using free private subnet 192.168.61.0/24: &{IP:192.168.61.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.61.0/24 Gateway:192.168.61.1 ClientMin:192.168.61.2 ClientMax:192.168.61.254 Broadcast:192.168.61.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0002ca4e0}
I0319 20:25:20.039509 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | created network xml:
I0319 20:25:20.039532 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <network>
I0319 20:25:20.039545 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <name>mk-old-k8s-version-159022</name>
I0319 20:25:20.039562 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <dns enable='no'/>
I0319 20:25:20.039575 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG |
I0319 20:25:20.039585 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <ip address='192.168.61.1' netmask='255.255.255.0'>
I0319 20:25:20.039598 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <dhcp>
I0319 20:25:20.039616 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | <range start='192.168.61.2' end='192.168.61.253'/>
I0319 20:25:20.039626 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | </dhcp>
I0319 20:25:20.039633 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | </ip>
I0319 20:25:20.039641 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG |
I0319 20:25:20.039648 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | </network>
I0319 20:25:20.039658 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG |
I0319 20:25:20.044896 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | trying to create private KVM network mk-old-k8s-version-159022 192.168.61.0/24...
I0319 20:25:20.113584 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | private KVM network mk-old-k8s-version-159022 192.168.61.0/24 created
I0319 20:25:20.113614 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting up store path in /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022 ...
I0319 20:25:20.113628 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.113550 56546 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/18453-10028/.minikube
I0319 20:25:20.113655 55982 main.go:141] libmachine: (old-k8s-version-159022) Building disk image from file:///home/jenkins/minikube-integration/18453-10028/.minikube/cache/iso/amd64/minikube-v1.32.1-1710573846-18277-amd64.iso
I0319 20:25:20.113670 55982 main.go:141] libmachine: (old-k8s-version-159022) Downloading /home/jenkins/minikube-integration/18453-10028/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/18453-10028/.minikube/cache/iso/amd64/minikube-v1.32.1-1710573846-18277-amd64.iso...
I0319 20:25:20.346103 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.345974 56546 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa...
I0319 20:25:20.422890 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.422761 56546 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/old-k8s-version-159022.rawdisk...
I0319 20:25:20.422926 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Writing magic tar header
I0319 20:25:20.422944 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Writing SSH key tar header
I0319 20:25:20.423104 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:20.422988 56546 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022 ...
I0319 20:25:20.423169 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022 (perms=drwx------)
I0319 20:25:20.423184 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022
I0319 20:25:20.423201 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins/minikube-integration/18453-10028/.minikube/machines (perms=drwxr-xr-x)
I0319 20:25:20.423217 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins/minikube-integration/18453-10028/.minikube (perms=drwxr-xr-x)
I0319 20:25:20.423230 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18453-10028/.minikube/machines
I0319 20:25:20.423236 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins/minikube-integration/18453-10028 (perms=drwxrwxr-x)
I0319 20:25:20.423247 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0319 20:25:20.423260 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18453-10028/.minikube
I0319 20:25:20.423274 55982 main.go:141] libmachine: (old-k8s-version-159022) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0319 20:25:20.423291 55982 main.go:141] libmachine: (old-k8s-version-159022) Creating domain...
I0319 20:25:20.423307 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18453-10028
I0319 20:25:20.423316 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0319 20:25:20.423322 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home/jenkins
I0319 20:25:20.423333 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Checking permissions on dir: /home
I0319 20:25:20.423345 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Skipping /home - not owner
I0319 20:25:20.424469 55982 main.go:141] libmachine: (old-k8s-version-159022) define libvirt domain using xml:
I0319 20:25:20.424484 55982 main.go:141] libmachine: (old-k8s-version-159022) <domain type='kvm'>
I0319 20:25:20.424492 55982 main.go:141] libmachine: (old-k8s-version-159022) <name>old-k8s-version-159022</name>
I0319 20:25:20.424500 55982 main.go:141] libmachine: (old-k8s-version-159022) <memory unit='MiB'>2200</memory>
I0319 20:25:20.424505 55982 main.go:141] libmachine: (old-k8s-version-159022) <vcpu>2</vcpu>
I0319 20:25:20.424510 55982 main.go:141] libmachine: (old-k8s-version-159022) <features>
I0319 20:25:20.424517 55982 main.go:141] libmachine: (old-k8s-version-159022) <acpi/>
I0319 20:25:20.424536 55982 main.go:141] libmachine: (old-k8s-version-159022) <apic/>
I0319 20:25:20.424549 55982 main.go:141] libmachine: (old-k8s-version-159022) <pae/>
I0319 20:25:20.424569 55982 main.go:141] libmachine: (old-k8s-version-159022)
I0319 20:25:20.424581 55982 main.go:141] libmachine: (old-k8s-version-159022) </features>
I0319 20:25:20.424596 55982 main.go:141] libmachine: (old-k8s-version-159022) <cpu mode='host-passthrough'>
I0319 20:25:20.424607 55982 main.go:141] libmachine: (old-k8s-version-159022)
I0319 20:25:20.424614 55982 main.go:141] libmachine: (old-k8s-version-159022) </cpu>
I0319 20:25:20.424625 55982 main.go:141] libmachine: (old-k8s-version-159022) <os>
I0319 20:25:20.424635 55982 main.go:141] libmachine: (old-k8s-version-159022) <type>hvm</type>
I0319 20:25:20.424661 55982 main.go:141] libmachine: (old-k8s-version-159022) <boot dev='cdrom'/>
I0319 20:25:20.424684 55982 main.go:141] libmachine: (old-k8s-version-159022) <boot dev='hd'/>
I0319 20:25:20.424706 55982 main.go:141] libmachine: (old-k8s-version-159022) <bootmenu enable='no'/>
I0319 20:25:20.424721 55982 main.go:141] libmachine: (old-k8s-version-159022) </os>
I0319 20:25:20.424734 55982 main.go:141] libmachine: (old-k8s-version-159022) <devices>
I0319 20:25:20.424742 55982 main.go:141] libmachine: (old-k8s-version-159022) <disk type='file' device='cdrom'>
I0319 20:25:20.424759 55982 main.go:141] libmachine: (old-k8s-version-159022) <source file='/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/boot2docker.iso'/>
I0319 20:25:20.424778 55982 main.go:141] libmachine: (old-k8s-version-159022) <target dev='hdc' bus='scsi'/>
I0319 20:25:20.424788 55982 main.go:141] libmachine: (old-k8s-version-159022) <readonly/>
I0319 20:25:20.424797 55982 main.go:141] libmachine: (old-k8s-version-159022) </disk>
I0319 20:25:20.424807 55982 main.go:141] libmachine: (old-k8s-version-159022) <disk type='file' device='disk'>
I0319 20:25:20.424824 55982 main.go:141] libmachine: (old-k8s-version-159022) <driver name='qemu' type='raw' cache='default' io='threads' />
I0319 20:25:20.424842 55982 main.go:141] libmachine: (old-k8s-version-159022) <source file='/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/old-k8s-version-159022.rawdisk'/>
I0319 20:25:20.424853 55982 main.go:141] libmachine: (old-k8s-version-159022) <target dev='hda' bus='virtio'/>
I0319 20:25:20.424869 55982 main.go:141] libmachine: (old-k8s-version-159022) </disk>
I0319 20:25:20.424881 55982 main.go:141] libmachine: (old-k8s-version-159022) <interface type='network'>
I0319 20:25:20.424892 55982 main.go:141] libmachine: (old-k8s-version-159022) <source network='mk-old-k8s-version-159022'/>
I0319 20:25:20.424907 55982 main.go:141] libmachine: (old-k8s-version-159022) <model type='virtio'/>
I0319 20:25:20.424919 55982 main.go:141] libmachine: (old-k8s-version-159022) </interface>
I0319 20:25:20.424926 55982 main.go:141] libmachine: (old-k8s-version-159022) <interface type='network'>
I0319 20:25:20.424947 55982 main.go:141] libmachine: (old-k8s-version-159022) <source network='default'/>
I0319 20:25:20.424967 55982 main.go:141] libmachine: (old-k8s-version-159022) <model type='virtio'/>
I0319 20:25:20.424977 55982 main.go:141] libmachine: (old-k8s-version-159022) </interface>
I0319 20:25:20.424984 55982 main.go:141] libmachine: (old-k8s-version-159022) <serial type='pty'>
I0319 20:25:20.425000 55982 main.go:141] libmachine: (old-k8s-version-159022) <target port='0'/>
I0319 20:25:20.425011 55982 main.go:141] libmachine: (old-k8s-version-159022) </serial>
I0319 20:25:20.425021 55982 main.go:141] libmachine: (old-k8s-version-159022) <console type='pty'>
I0319 20:25:20.425033 55982 main.go:141] libmachine: (old-k8s-version-159022) <target type='serial' port='0'/>
I0319 20:25:20.425044 55982 main.go:141] libmachine: (old-k8s-version-159022) </console>
I0319 20:25:20.425054 55982 main.go:141] libmachine: (old-k8s-version-159022) <rng model='virtio'>
I0319 20:25:20.425074 55982 main.go:141] libmachine: (old-k8s-version-159022) <backend model='random'>/dev/random</backend>
I0319 20:25:20.425088 55982 main.go:141] libmachine: (old-k8s-version-159022) </rng>
I0319 20:25:20.425096 55982 main.go:141] libmachine: (old-k8s-version-159022)
I0319 20:25:20.425114 55982 main.go:141] libmachine: (old-k8s-version-159022)
I0319 20:25:20.425127 55982 main.go:141] libmachine: (old-k8s-version-159022) </devices>
I0319 20:25:20.425137 55982 main.go:141] libmachine: (old-k8s-version-159022) </domain>
I0319 20:25:20.425146 55982 main.go:141] libmachine: (old-k8s-version-159022)
I0319 20:25:20.429121 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:50:b9:64 in network default
I0319 20:25:20.429776 55982 main.go:141] libmachine: (old-k8s-version-159022) Ensuring networks are active...
I0319 20:25:20.429803 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:20.430475 55982 main.go:141] libmachine: (old-k8s-version-159022) Ensuring network default is active
I0319 20:25:20.430840 55982 main.go:141] libmachine: (old-k8s-version-159022) Ensuring network mk-old-k8s-version-159022 is active
I0319 20:25:20.431352 55982 main.go:141] libmachine: (old-k8s-version-159022) Getting domain xml...
I0319 20:25:20.432067 55982 main.go:141] libmachine: (old-k8s-version-159022) Creating domain...
I0319 20:25:21.692285 55982 main.go:141] libmachine: (old-k8s-version-159022) Waiting to get IP...
I0319 20:25:21.693038 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:21.693483 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:21.693531 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:21.693478 56546 retry.go:31] will retry after 300.974755ms: waiting for machine to come up
I0319 20:25:21.996216 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:21.996815 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:21.996846 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:21.996757 56546 retry.go:31] will retry after 378.350693ms: waiting for machine to come up
I0319 20:25:22.377082 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:22.378062 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:22.378093 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:22.378011 56546 retry.go:31] will retry after 337.090678ms: waiting for machine to come up
I0319 20:25:22.716618 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:22.717084 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:22.717108 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:22.717034 56546 retry.go:31] will retry after 448.487874ms: waiting for machine to come up
I0319 20:25:23.167271 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:23.167958 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:23.167989 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:23.167909 56546 retry.go:31] will retry after 738.736662ms: waiting for machine to come up
I0319 20:25:23.907682 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:23.908392 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:23.908420 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:23.908359 56546 retry.go:31] will retry after 823.841957ms: waiting for machine to come up
I0319 20:25:24.734060 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:24.734560 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:24.734588 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:24.734505 56546 retry.go:31] will retry after 1.015139108s: waiting for machine to come up
I0319 20:25:25.751162 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:25.751729 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:25.751760 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:25.751675 56546 retry.go:31] will retry after 901.716648ms: waiting for machine to come up
I0319 20:25:26.654593 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:26.655089 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:26.655166 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:26.655062 56546 retry.go:31] will retry after 1.819645561s: waiting for machine to come up
I0319 20:25:28.475818 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:28.476348 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:28.476378 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:28.476308 56546 retry.go:31] will retry after 1.820594289s: waiting for machine to come up
I0319 20:25:30.298258 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:30.298796 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:30.298831 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:30.298734 56546 retry.go:31] will retry after 2.616805696s: waiting for machine to come up
I0319 20:25:32.918305 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:32.918846 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:32.918877 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:32.918787 56546 retry.go:31] will retry after 3.429736925s: waiting for machine to come up
I0319 20:25:36.350814 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:36.351305 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:36.351325 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:36.351269 56546 retry.go:31] will retry after 4.231400763s: waiting for machine to come up
I0319 20:25:40.584325 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:40.584739 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find current IP address of domain old-k8s-version-159022 in network mk-old-k8s-version-159022
I0319 20:25:40.584767 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | I0319 20:25:40.584692 56546 retry.go:31] will retry after 5.452618525s: waiting for machine to come up
I0319 20:25:46.038847 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.039345 55982 main.go:141] libmachine: (old-k8s-version-159022) Found IP for machine: 192.168.61.28
I0319 20:25:46.039379 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has current primary IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.039389 55982 main.go:141] libmachine: (old-k8s-version-159022) Reserving static IP address...
I0319 20:25:46.039719 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-159022", mac: "52:54:00:be:83:01", ip: "192.168.61.28"} in network mk-old-k8s-version-159022
I0319 20:25:46.112140 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Getting to WaitForSSH function...
I0319 20:25:46.112168 55982 main.go:141] libmachine: (old-k8s-version-159022) Reserved static IP address: 192.168.61.28
I0319 20:25:46.112182 55982 main.go:141] libmachine: (old-k8s-version-159022) Waiting for SSH to be available...
I0319 20:25:46.114965 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.115446 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:minikube Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.115479 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.115614 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Using SSH client type: external
I0319 20:25:46.115639 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Using SSH private key: /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa (-rw-------)
I0319 20:25:46.115668 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.61.28 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa -p 22] /usr/bin/ssh <nil>}
I0319 20:25:46.115689 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | About to run SSH command:
I0319 20:25:46.115704 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | exit 0
I0319 20:25:46.249138 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | SSH cmd err, output: <nil>:
I0319 20:25:46.249424 55982 main.go:141] libmachine: (old-k8s-version-159022) KVM machine creation complete!
I0319 20:25:46.249807 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetConfigRaw
I0319 20:25:46.250468 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:46.250692 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:46.250880 55982 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0319 20:25:46.250898 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetState
I0319 20:25:46.252136 55982 main.go:141] libmachine: Detecting operating system of created instance...
I0319 20:25:46.252148 55982 main.go:141] libmachine: Waiting for SSH to be available...
I0319 20:25:46.252153 55982 main.go:141] libmachine: Getting to WaitForSSH function...
I0319 20:25:46.252159 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.254582 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.255005 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.255035 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.255178 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:46.255401 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.255543 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.255708 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:46.255914 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:46.256152 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:46.256167 55982 main.go:141] libmachine: About to run SSH command:
exit 0
I0319 20:25:46.372010 55982 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0319 20:25:46.372034 55982 main.go:141] libmachine: Detecting the provisioner...
I0319 20:25:46.372044 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.374984 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.375360 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.375389 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.375524 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:46.375718 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.375883 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.376015 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:46.376152 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:46.376351 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:46.376367 55982 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0319 20:25:46.494038 55982 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0319 20:25:46.494105 55982 main.go:141] libmachine: found compatible host: buildroot
I0319 20:25:46.494118 55982 main.go:141] libmachine: Provisioning with buildroot...
I0319 20:25:46.494132 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetMachineName
I0319 20:25:46.494390 55982 buildroot.go:166] provisioning hostname "old-k8s-version-159022"
I0319 20:25:46.494419 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetMachineName
I0319 20:25:46.494663 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.497351 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.497668 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.497697 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.497804 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:46.497978 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.498135 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.498270 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:46.498444 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:46.498603 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:46.498615 55982 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-159022 && echo "old-k8s-version-159022" | sudo tee /etc/hostname
I0319 20:25:46.628943 55982 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-159022
I0319 20:25:46.628978 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.631797 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.632186 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.632209 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.632454 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:46.632664 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.632822 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.633003 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:46.633200 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:46.633371 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:46.633387 55982 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-159022' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-159022/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-159022' | sudo tee -a /etc/hosts;
fi
fi
I0319 20:25:46.761522 55982 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0319 20:25:46.761556 55982 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/18453-10028/.minikube CaCertPath:/home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18453-10028/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18453-10028/.minikube}
I0319 20:25:46.761592 55982 buildroot.go:174] setting up certificates
I0319 20:25:46.761606 55982 provision.go:84] configureAuth start
I0319 20:25:46.761617 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetMachineName
I0319 20:25:46.761884 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetIP
I0319 20:25:46.764704 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.765058 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.765089 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.765274 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.767545 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.767887 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.767923 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.768048 55982 provision.go:143] copyHostCerts
I0319 20:25:46.768113 55982 exec_runner.go:144] found /home/jenkins/minikube-integration/18453-10028/.minikube/ca.pem, removing ...
I0319 20:25:46.768127 55982 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18453-10028/.minikube/ca.pem
I0319 20:25:46.768208 55982 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18453-10028/.minikube/ca.pem (1082 bytes)
I0319 20:25:46.768347 55982 exec_runner.go:144] found /home/jenkins/minikube-integration/18453-10028/.minikube/cert.pem, removing ...
I0319 20:25:46.768364 55982 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18453-10028/.minikube/cert.pem
I0319 20:25:46.768397 55982 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18453-10028/.minikube/cert.pem (1123 bytes)
I0319 20:25:46.768513 55982 exec_runner.go:144] found /home/jenkins/minikube-integration/18453-10028/.minikube/key.pem, removing ...
I0319 20:25:46.768524 55982 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18453-10028/.minikube/key.pem
I0319 20:25:46.768553 55982 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18453-10028/.minikube/key.pem (1675 bytes)
I0319 20:25:46.768642 55982 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18453-10028/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-159022 san=[127.0.0.1 192.168.61.28 localhost minikube old-k8s-version-159022]
I0319 20:25:46.906018 55982 provision.go:177] copyRemoteCerts
I0319 20:25:46.906097 55982 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0319 20:25:46.906126 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:46.908825 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.909301 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:46.909328 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:46.909399 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:46.909611 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:46.909795 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:46.909933 55982 sshutil.go:53] new ssh client: &{IP:192.168.61.28 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa Username:docker}
I0319 20:25:47.001445 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0319 20:25:47.032497 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0319 20:25:47.062999 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0319 20:25:47.089974 55982 provision.go:87] duration metric: took 328.354781ms to configureAuth
I0319 20:25:47.090004 55982 buildroot.go:189] setting minikube options for container-runtime
I0319 20:25:47.090215 55982 config.go:182] Loaded profile config "old-k8s-version-159022": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0319 20:25:47.090290 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:47.092919 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.093301 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.093329 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.093482 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:47.093671 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.093834 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.094009 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:47.094188 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:47.094361 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:47.094389 55982 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0319 20:25:47.389752 55982 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0319 20:25:47.389778 55982 main.go:141] libmachine: Checking connection to Docker...
I0319 20:25:47.389787 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetURL
I0319 20:25:47.391023 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | Using libvirt version 6000000
I0319 20:25:47.393502 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.393873 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.393918 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.394056 55982 main.go:141] libmachine: Docker is up and running!
I0319 20:25:47.394073 55982 main.go:141] libmachine: Reticulating splines...
I0319 20:25:47.394081 55982 client.go:171] duration metric: took 27.359310019s to LocalClient.Create
I0319 20:25:47.394110 55982 start.go:167] duration metric: took 27.359377921s to libmachine.API.Create "old-k8s-version-159022"
I0319 20:25:47.394120 55982 start.go:293] postStartSetup for "old-k8s-version-159022" (driver="kvm2")
I0319 20:25:47.394131 55982 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0319 20:25:47.394146 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:47.394369 55982 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0319 20:25:47.394393 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:47.396807 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.397109 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.397143 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.397295 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:47.397448 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.397599 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:47.397785 55982 sshutil.go:53] new ssh client: &{IP:192.168.61.28 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa Username:docker}
I0319 20:25:47.490331 55982 ssh_runner.go:195] Run: cat /etc/os-release
I0319 20:25:47.495270 55982 info.go:137] Remote host: Buildroot 2023.02.9
I0319 20:25:47.495296 55982 filesync.go:126] Scanning /home/jenkins/minikube-integration/18453-10028/.minikube/addons for local assets ...
I0319 20:25:47.495364 55982 filesync.go:126] Scanning /home/jenkins/minikube-integration/18453-10028/.minikube/files for local assets ...
I0319 20:25:47.495431 55982 filesync.go:149] local asset: /home/jenkins/minikube-integration/18453-10028/.minikube/files/etc/ssl/certs/173012.pem -> 173012.pem in /etc/ssl/certs
I0319 20:25:47.495517 55982 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0319 20:25:47.508162 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/files/etc/ssl/certs/173012.pem --> /etc/ssl/certs/173012.pem (1708 bytes)
I0319 20:25:47.536042 55982 start.go:296] duration metric: took 141.908819ms for postStartSetup
I0319 20:25:47.536094 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetConfigRaw
I0319 20:25:47.536673 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetIP
I0319 20:25:47.539443 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.539769 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.539793 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.540012 55982 profile.go:142] Saving config to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/config.json ...
I0319 20:25:47.540247 55982 start.go:128] duration metric: took 27.526518248s to createHost
I0319 20:25:47.540294 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:47.542495 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.542838 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.542860 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.543017 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:47.543167 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.543310 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.543428 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:47.543574 55982 main.go:141] libmachine: Using SSH client type: native
I0319 20:25:47.543732 55982 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.61.28 22 <nil> <nil>}
I0319 20:25:47.543745 55982 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0319 20:25:47.661671 55982 main.go:141] libmachine: SSH cmd err, output: <nil>: 1710879947.645733085
I0319 20:25:47.661695 55982 fix.go:216] guest clock: 1710879947.645733085
I0319 20:25:47.661702 55982 fix.go:229] Guest: 2024-03-19 20:25:47.645733085 +0000 UTC Remote: 2024-03-19 20:25:47.540279791 +0000 UTC m=+61.265095128 (delta=105.453294ms)
I0319 20:25:47.661719 55982 fix.go:200] guest clock delta is within tolerance: 105.453294ms
I0319 20:25:47.661723 55982 start.go:83] releasing machines lock for "old-k8s-version-159022", held for 27.648182773s
I0319 20:25:47.661747 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:47.662043 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetIP
I0319 20:25:47.665318 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.665745 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.665772 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.665924 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:47.666372 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:47.666626 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .DriverName
I0319 20:25:47.666722 55982 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0319 20:25:47.666765 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:47.666857 55982 ssh_runner.go:195] Run: cat /version.json
I0319 20:25:47.666887 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHHostname
I0319 20:25:47.669670 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.669921 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.670174 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.670205 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.670519 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:47.670533 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:47.670614 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:47.670739 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.670765 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHPort
I0319 20:25:47.670897 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:47.670944 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHKeyPath
I0319 20:25:47.671104 55982 sshutil.go:53] new ssh client: &{IP:192.168.61.28 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa Username:docker}
I0319 20:25:47.671131 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetSSHUsername
I0319 20:25:47.671276 55982 sshutil.go:53] new ssh client: &{IP:192.168.61.28 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18453-10028/.minikube/machines/old-k8s-version-159022/id_rsa Username:docker}
I0319 20:25:47.762533 55982 ssh_runner.go:195] Run: systemctl --version
I0319 20:25:47.788473 55982 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0319 20:25:47.954051 55982 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0319 20:25:47.962242 55982 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0319 20:25:47.962319 55982 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0319 20:25:47.982103 55982 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0319 20:25:47.982145 55982 start.go:494] detecting cgroup driver to use...
I0319 20:25:47.982217 55982 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0319 20:25:48.003993 55982 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0319 20:25:48.021105 55982 docker.go:217] disabling cri-docker service (if available) ...
I0319 20:25:48.021166 55982 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0319 20:25:48.040411 55982 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0319 20:25:48.058121 55982 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0319 20:25:48.212569 55982 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0319 20:25:48.390385 55982 docker.go:233] disabling docker service ...
I0319 20:25:48.390453 55982 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0319 20:25:48.408015 55982 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0319 20:25:48.424140 55982 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0319 20:25:48.566359 55982 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0319 20:25:48.725824 55982 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0319 20:25:48.754844 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0319 20:25:48.782110 55982 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0319 20:25:48.782179 55982 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0319 20:25:48.795927 55982 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0319 20:25:48.795985 55982 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0319 20:25:48.807923 55982 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0319 20:25:48.819776 55982 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0319 20:25:48.831705 55982 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0319 20:25:48.845768 55982 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0319 20:25:48.862637 55982 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0319 20:25:48.862690 55982 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0319 20:25:48.880204 55982 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0319 20:25:48.894624 55982 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0319 20:25:49.054962 55982 ssh_runner.go:195] Run: sudo systemctl restart crio
I0319 20:25:49.214429 55982 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0319 20:25:49.214492 55982 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0319 20:25:49.219878 55982 start.go:562] Will wait 60s for crictl version
I0319 20:25:49.219932 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:49.224417 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0319 20:25:49.275717 55982 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0319 20:25:49.275794 55982 ssh_runner.go:195] Run: crio --version
I0319 20:25:49.313888 55982 ssh_runner.go:195] Run: crio --version
I0319 20:25:49.358265 55982 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0319 20:25:49.359618 55982 main.go:141] libmachine: (old-k8s-version-159022) Calling .GetIP
I0319 20:25:49.362790 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:49.363262 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:be:83:01", ip: ""} in network mk-old-k8s-version-159022: {Iface:virbr3 ExpiryTime:2024-03-19 21:25:36 +0000 UTC Type:0 Mac:52:54:00:be:83:01 Iaid: IPaddr:192.168.61.28 Prefix:24 Hostname:old-k8s-version-159022 Clientid:01:52:54:00:be:83:01}
I0319 20:25:49.363293 55982 main.go:141] libmachine: (old-k8s-version-159022) DBG | domain old-k8s-version-159022 has defined IP address 192.168.61.28 and MAC address 52:54:00:be:83:01 in network mk-old-k8s-version-159022
I0319 20:25:49.363591 55982 ssh_runner.go:195] Run: grep 192.168.61.1 host.minikube.internal$ /etc/hosts
I0319 20:25:49.369067 55982 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.61.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0319 20:25:49.385009 55982 kubeadm.go:877] updating cluster {Name:old-k8s-version-159022 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18277/minikube-v1.32.1-1710573846-18277-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-159022 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.61.28 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0319 20:25:49.385178 55982 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0319 20:25:49.385239 55982 ssh_runner.go:195] Run: sudo crictl images --output json
I0319 20:25:49.435514 55982 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0319 20:25:49.435615 55982 ssh_runner.go:195] Run: which lz4
I0319 20:25:49.441041 55982 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0319 20:25:49.446341 55982 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0319 20:25:49.446371 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0319 20:25:51.708334 55982 crio.go:462] duration metric: took 2.267338917s to copy over tarball
I0319 20:25:51.708422 55982 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0319 20:25:54.900870 55982 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (3.192420858s)
I0319 20:25:54.900899 55982 crio.go:469] duration metric: took 3.192525975s to extract the tarball
I0319 20:25:54.900908 55982 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0319 20:25:54.945654 55982 ssh_runner.go:195] Run: sudo crictl images --output json
I0319 20:25:54.997150 55982 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0319 20:25:54.997184 55982 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0319 20:25:54.997296 55982 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0319 20:25:54.997579 55982 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0319 20:25:54.997591 55982 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0319 20:25:54.997620 55982 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0319 20:25:54.997723 55982 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0319 20:25:54.997743 55982 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0319 20:25:54.997837 55982 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0319 20:25:54.997728 55982 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0319 20:25:54.998942 55982 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0319 20:25:54.999189 55982 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0319 20:25:54.999209 55982 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0319 20:25:54.999212 55982 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0319 20:25:54.999217 55982 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0319 20:25:54.999298 55982 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0319 20:25:54.999308 55982 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0319 20:25:54.999788 55982 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0319 20:25:55.154901 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0319 20:25:55.178995 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0319 20:25:55.194425 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0319 20:25:55.207142 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0319 20:25:55.213949 55982 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0319 20:25:55.213982 55982 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0319 20:25:55.214022 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.215693 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0319 20:25:55.249431 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0319 20:25:55.295990 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0319 20:25:55.300329 55982 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0319 20:25:55.300435 55982 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0319 20:25:55.300501 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.300329 55982 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0319 20:25:55.300546 55982 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0319 20:25:55.300605 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.346880 55982 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0319 20:25:55.346948 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0319 20:25:55.346976 55982 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0319 20:25:55.347016 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.369140 55982 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0319 20:25:55.369184 55982 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0319 20:25:55.369244 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.406307 55982 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0319 20:25:55.406354 55982 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0319 20:25:55.406381 55982 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0319 20:25:55.406406 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0319 20:25:55.406422 55982 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0319 20:25:55.406467 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.406483 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0319 20:25:55.406411 55982 ssh_runner.go:195] Run: which crictl
I0319 20:25:55.437222 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0319 20:25:55.437309 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0319 20:25:55.437341 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0319 20:25:55.517473 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0319 20:25:55.517517 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0319 20:25:55.517577 55982 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0319 20:25:55.517723 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0319 20:25:55.553853 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0319 20:25:55.561362 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0319 20:25:55.598715 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0319 20:25:55.598735 55982 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0319 20:25:55.931464 55982 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0319 20:25:56.081504 55982 cache_images.go:92] duration metric: took 1.084303153s to LoadCachedImages
W0319 20:25:56.081622 55982 out.go:239] X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18453-10028/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0319 20:25:56.081643 55982 kubeadm.go:928] updating node { 192.168.61.28 8443 v1.20.0 crio true true} ...
I0319 20:25:56.081776 55982 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-159022 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.61.28
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-159022 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0319 20:25:56.081863 55982 ssh_runner.go:195] Run: crio config
I0319 20:25:56.139415 55982 cni.go:84] Creating CNI manager for ""
I0319 20:25:56.139438 55982 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0319 20:25:56.139450 55982 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0319 20:25:56.139467 55982 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.61.28 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-159022 NodeName:old-k8s-version-159022 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.61.28"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.61.28 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0319 20:25:56.139666 55982 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.61.28
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-159022"
kubeletExtraArgs:
node-ip: 192.168.61.28
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.61.28"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0319 20:25:56.139756 55982 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0319 20:25:56.151535 55982 binaries.go:44] Found k8s binaries, skipping transfer
I0319 20:25:56.151615 55982 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0319 20:25:56.162688 55982 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0319 20:25:56.183525 55982 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0319 20:25:56.202662 55982 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0319 20:25:56.222091 55982 ssh_runner.go:195] Run: grep 192.168.61.28 control-plane.minikube.internal$ /etc/hosts
I0319 20:25:56.226617 55982 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.61.28 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0319 20:25:56.240650 55982 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0319 20:25:56.387390 55982 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0319 20:25:56.579447 55982 certs.go:68] Setting up /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022 for IP: 192.168.61.28
I0319 20:25:56.579475 55982 certs.go:194] generating shared ca certs ...
I0319 20:25:56.579495 55982 certs.go:226] acquiring lock for ca certs: {Name:mk753cf639d289f9668854fb4e0e29364ad184fb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:56.579653 55982 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18453-10028/.minikube/ca.key
I0319 20:25:56.579726 55982 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18453-10028/.minikube/proxy-client-ca.key
I0319 20:25:56.579755 55982 certs.go:256] generating profile certs ...
I0319 20:25:56.579866 55982 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.key
I0319 20:25:56.579886 55982 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.crt with IP's: []
I0319 20:25:56.671840 55982 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.crt ...
I0319 20:25:56.671869 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.crt: {Name:mk61c72fce9c679651e7a9e1decdd5e5de4586de Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:56.672041 55982 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.key ...
I0319 20:25:56.672059 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/client.key: {Name:mk4f4dae15ac74583de7977e4327a5ef8cb539c9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:56.672176 55982 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key.d78c40b4
I0319 20:25:56.672196 55982 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt.d78c40b4 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.61.28]
I0319 20:25:56.842042 55982 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt.d78c40b4 ...
I0319 20:25:56.842071 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt.d78c40b4: {Name:mkca57b1829959d481c3f86e2a09caa6cc12fe28 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:56.842247 55982 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key.d78c40b4 ...
I0319 20:25:56.842263 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key.d78c40b4: {Name:mk2b34b2cf48017b7663a3ecf0d75ff3f102a0e5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:56.842355 55982 certs.go:381] copying /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt.d78c40b4 -> /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt
I0319 20:25:56.842468 55982 certs.go:385] copying /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key.d78c40b4 -> /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key
I0319 20:25:56.842550 55982 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.key
I0319 20:25:56.842576 55982 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.crt with IP's: []
I0319 20:25:57.176440 55982 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.crt ...
I0319 20:25:57.176468 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.crt: {Name:mk6cfea44b706848d0ef5c66bf794a61fff6c263 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:57.176654 55982 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.key ...
I0319 20:25:57.176673 55982 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.key: {Name:mkac062ea72fce040eaa4b5cd499c8a1bc2a3b3b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0319 20:25:57.176863 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/17301.pem (1338 bytes)
W0319 20:25:57.176914 55982 certs.go:480] ignoring /home/jenkins/minikube-integration/18453-10028/.minikube/certs/17301_empty.pem, impossibly tiny 0 bytes
I0319 20:25:57.176924 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca-key.pem (1679 bytes)
I0319 20:25:57.176943 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/ca.pem (1082 bytes)
I0319 20:25:57.176967 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/cert.pem (1123 bytes)
I0319 20:25:57.176990 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/certs/key.pem (1675 bytes)
I0319 20:25:57.177026 55982 certs.go:484] found cert: /home/jenkins/minikube-integration/18453-10028/.minikube/files/etc/ssl/certs/173012.pem (1708 bytes)
I0319 20:25:57.177598 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0319 20:25:57.214307 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0319 20:25:57.242380 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0319 20:25:57.270810 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0319 20:25:57.298840 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0319 20:25:57.327424 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0319 20:25:57.354944 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0319 20:25:57.383921 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/profiles/old-k8s-version-159022/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0319 20:25:57.415824 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0319 20:25:57.446943 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/certs/17301.pem --> /usr/share/ca-certificates/17301.pem (1338 bytes)
I0319 20:25:57.498524 55982 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18453-10028/.minikube/files/etc/ssl/certs/173012.pem --> /usr/share/ca-certificates/173012.pem (1708 bytes)
I0319 20:25:57.524299 55982 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0319 20:25:57.548071 55982 ssh_runner.go:195] Run: openssl version
I0319 20:25:57.555174 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/17301.pem && ln -fs /usr/share/ca-certificates/17301.pem /etc/ssl/certs/17301.pem"
I0319 20:25:57.568658 55982 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/17301.pem
I0319 20:25:57.573925 55982 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 19 19:17 /usr/share/ca-certificates/17301.pem
I0319 20:25:57.573975 55982 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/17301.pem
I0319 20:25:57.580585 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/17301.pem /etc/ssl/certs/51391683.0"
I0319 20:25:57.593587 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/173012.pem && ln -fs /usr/share/ca-certificates/173012.pem /etc/ssl/certs/173012.pem"
I0319 20:25:57.606796 55982 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/173012.pem
I0319 20:25:57.612138 55982 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 19 19:17 /usr/share/ca-certificates/173012.pem
I0319 20:25:57.612195 55982 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/173012.pem
I0319 20:25:57.619084 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/173012.pem /etc/ssl/certs/3ec20f2e.0"
I0319 20:25:57.632991 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0319 20:25:57.645667 55982 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0319 20:25:57.650822 55982 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 19 19:07 /usr/share/ca-certificates/minikubeCA.pem
I0319 20:25:57.650871 55982 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0319 20:25:57.657697 55982 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0319 20:25:57.671652 55982 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0319 20:25:57.678066 55982 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0319 20:25:57.678131 55982 kubeadm.go:391] StartCluster: {Name:old-k8s-version-159022 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18277/minikube-v1.32.1-1710573846-18277-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1710284843-18375@sha256:d67c38c9fc2ad14c48d95e17cbac49314325db5758d8f7b3de60b927e62ce94f Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-159022 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.61.28 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0319 20:25:57.678227 55982 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0319 20:25:57.678281 55982 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0319 20:25:57.719280 55982 cri.go:89] found id: ""
I0319 20:25:57.719367 55982 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0319 20:25:57.731073 55982 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0319 20:25:57.742226 55982 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0319 20:25:57.755538 55982 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0319 20:25:57.755557 55982 kubeadm.go:156] found existing configuration files:
I0319 20:25:57.755603 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0319 20:25:57.767830 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0319 20:25:57.767899 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0319 20:25:57.778642 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0319 20:25:57.790544 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0319 20:25:57.790619 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0319 20:25:57.802730 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0319 20:25:57.813705 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0319 20:25:57.813756 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0319 20:25:57.825964 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0319 20:25:57.837099 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0319 20:25:57.837150 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0319 20:25:57.848351 55982 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0319 20:25:57.987481 55982 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0319 20:25:57.987556 55982 kubeadm.go:309] [preflight] Running pre-flight checks
I0319 20:25:58.185352 55982 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0319 20:25:58.185497 55982 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0319 20:25:58.185618 55982 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0319 20:25:58.461759 55982 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0319 20:25:58.464609 55982 out.go:204] - Generating certificates and keys ...
I0319 20:25:58.464729 55982 kubeadm.go:309] [certs] Using existing ca certificate authority
I0319 20:25:58.464818 55982 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0319 20:25:58.819523 55982 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0319 20:25:59.010342 55982 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0319 20:25:59.289340 55982 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0319 20:25:59.481304 55982 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0319 20:25:59.772779 55982 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0319 20:25:59.772969 55982 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
I0319 20:26:00.302141 55982 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0319 20:26:00.302338 55982 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
I0319 20:26:00.676075 55982 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0319 20:26:00.924844 55982 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0319 20:26:01.100878 55982 kubeadm.go:309] [certs] Generating "sa" key and public key
I0319 20:26:01.100971 55982 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0319 20:26:01.215620 55982 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0319 20:26:01.525378 55982 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0319 20:26:01.790567 55982 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0319 20:26:02.183004 55982 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0319 20:26:02.199227 55982 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0319 20:26:02.200349 55982 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0319 20:26:02.200402 55982 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0319 20:26:02.346678 55982 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0319 20:26:02.348574 55982 out.go:204] - Booting up control plane ...
I0319 20:26:02.348716 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0319 20:26:02.356184 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0319 20:26:02.357288 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0319 20:26:02.358196 55982 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0319 20:26:02.362557 55982 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0319 20:26:42.361450 55982 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0319 20:26:42.361557 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:26:42.361770 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:26:47.362533 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:26:47.362763 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:26:57.363368 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:26:57.363647 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:27:17.364761 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:27:17.365042 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:27:57.365154 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:27:57.365686 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:27:57.365702 55982 kubeadm.go:309]
I0319 20:27:57.365795 55982 kubeadm.go:309] Unfortunately, an error has occurred:
I0319 20:27:57.365886 55982 kubeadm.go:309] timed out waiting for the condition
I0319 20:27:57.365897 55982 kubeadm.go:309]
I0319 20:27:57.365971 55982 kubeadm.go:309] This error is likely caused by:
I0319 20:27:57.366073 55982 kubeadm.go:309] - The kubelet is not running
I0319 20:27:57.366246 55982 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0319 20:27:57.366261 55982 kubeadm.go:309]
I0319 20:27:57.366399 55982 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0319 20:27:57.366459 55982 kubeadm.go:309] - 'systemctl status kubelet'
I0319 20:27:57.366503 55982 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0319 20:27:57.366511 55982 kubeadm.go:309]
I0319 20:27:57.366664 55982 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0319 20:27:57.366775 55982 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0319 20:27:57.366783 55982 kubeadm.go:309]
I0319 20:27:57.366922 55982 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0319 20:27:57.367042 55982 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0319 20:27:57.367191 55982 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0319 20:27:57.367362 55982 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0319 20:27:57.367403 55982 kubeadm.go:309]
I0319 20:27:57.367633 55982 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0319 20:27:57.367824 55982 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0319 20:27:57.368029 55982 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
W0319 20:27:57.368202 55982 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-159022] and IPs [192.168.61.28 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0319 20:27:57.368250 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0319 20:28:00.047920 55982 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force": (2.67964074s)
I0319 20:28:00.048003 55982 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0319 20:28:00.064905 55982 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0319 20:28:00.077445 55982 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0319 20:28:00.077469 55982 kubeadm.go:156] found existing configuration files:
I0319 20:28:00.077520 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0319 20:28:00.089987 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0319 20:28:00.090035 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0319 20:28:00.101687 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0319 20:28:00.111634 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0319 20:28:00.111678 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0319 20:28:00.123195 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0319 20:28:00.134315 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0319 20:28:00.134375 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0319 20:28:00.145918 55982 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0319 20:28:00.156950 55982 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0319 20:28:00.157012 55982 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0319 20:28:00.167775 55982 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0319 20:28:00.411165 55982 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0319 20:29:56.819531 55982 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0319 20:29:56.819659 55982 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0319 20:29:56.821663 55982 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0319 20:29:56.821758 55982 kubeadm.go:309] [preflight] Running pre-flight checks
I0319 20:29:56.821886 55982 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0319 20:29:56.822011 55982 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0319 20:29:56.822097 55982 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0319 20:29:56.822150 55982 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0319 20:29:56.824019 55982 out.go:204] - Generating certificates and keys ...
I0319 20:29:56.824086 55982 kubeadm.go:309] [certs] Using existing ca certificate authority
I0319 20:29:56.824139 55982 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0319 20:29:56.824217 55982 kubeadm.go:309] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0319 20:29:56.824322 55982 kubeadm.go:309] [certs] Using existing front-proxy-ca certificate authority
I0319 20:29:56.824432 55982 kubeadm.go:309] [certs] Using existing front-proxy-client certificate and key on disk
I0319 20:29:56.824525 55982 kubeadm.go:309] [certs] Using existing etcd/ca certificate authority
I0319 20:29:56.824622 55982 kubeadm.go:309] [certs] Using existing etcd/server certificate and key on disk
I0319 20:29:56.824697 55982 kubeadm.go:309] [certs] Using existing etcd/peer certificate and key on disk
I0319 20:29:56.824801 55982 kubeadm.go:309] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0319 20:29:56.824933 55982 kubeadm.go:309] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0319 20:29:56.824992 55982 kubeadm.go:309] [certs] Using the existing "sa" key
I0319 20:29:56.825047 55982 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0319 20:29:56.825094 55982 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0319 20:29:56.825139 55982 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0319 20:29:56.825207 55982 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0319 20:29:56.825259 55982 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0319 20:29:56.825368 55982 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0319 20:29:56.825465 55982 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0319 20:29:56.825517 55982 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0319 20:29:56.825616 55982 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0319 20:29:56.827311 55982 out.go:204] - Booting up control plane ...
I0319 20:29:56.827412 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0319 20:29:56.827498 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0319 20:29:56.827584 55982 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0319 20:29:56.827666 55982 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0319 20:29:56.827869 55982 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0319 20:29:56.827937 55982 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0319 20:29:56.828020 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:29:56.828203 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:29:56.828321 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:29:56.828530 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:29:56.828612 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:29:56.828876 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:29:56.828952 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:29:56.829121 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:29:56.829221 55982 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0319 20:29:56.829401 55982 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0319 20:29:56.829410 55982 kubeadm.go:309]
I0319 20:29:56.829463 55982 kubeadm.go:309] Unfortunately, an error has occurred:
I0319 20:29:56.829519 55982 kubeadm.go:309] timed out waiting for the condition
I0319 20:29:56.829530 55982 kubeadm.go:309]
I0319 20:29:56.829582 55982 kubeadm.go:309] This error is likely caused by:
I0319 20:29:56.829633 55982 kubeadm.go:309] - The kubelet is not running
I0319 20:29:56.829786 55982 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0319 20:29:56.829797 55982 kubeadm.go:309]
I0319 20:29:56.829947 55982 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0319 20:29:56.830007 55982 kubeadm.go:309] - 'systemctl status kubelet'
I0319 20:29:56.830054 55982 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0319 20:29:56.830064 55982 kubeadm.go:309]
I0319 20:29:56.830224 55982 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0319 20:29:56.830362 55982 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0319 20:29:56.830372 55982 kubeadm.go:309]
I0319 20:29:56.830458 55982 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0319 20:29:56.830558 55982 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0319 20:29:56.830652 55982 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0319 20:29:56.830715 55982 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0319 20:29:56.830741 55982 kubeadm.go:309]
I0319 20:29:56.830777 55982 kubeadm.go:393] duration metric: took 3m59.15265081s to StartCluster
I0319 20:29:56.830816 55982 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0319 20:29:56.830860 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0319 20:29:56.885490 55982 cri.go:89] found id: ""
I0319 20:29:56.885521 55982 logs.go:276] 0 containers: []
W0319 20:29:56.885533 55982 logs.go:278] No container was found matching "kube-apiserver"
I0319 20:29:56.885540 55982 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0319 20:29:56.885599 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0319 20:29:56.923850 55982 cri.go:89] found id: ""
I0319 20:29:56.923879 55982 logs.go:276] 0 containers: []
W0319 20:29:56.923889 55982 logs.go:278] No container was found matching "etcd"
I0319 20:29:56.923896 55982 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0319 20:29:56.923954 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0319 20:29:56.970642 55982 cri.go:89] found id: ""
I0319 20:29:56.970669 55982 logs.go:276] 0 containers: []
W0319 20:29:56.970679 55982 logs.go:278] No container was found matching "coredns"
I0319 20:29:56.970687 55982 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0319 20:29:56.970748 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0319 20:29:57.009555 55982 cri.go:89] found id: ""
I0319 20:29:57.009587 55982 logs.go:276] 0 containers: []
W0319 20:29:57.009597 55982 logs.go:278] No container was found matching "kube-scheduler"
I0319 20:29:57.009605 55982 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0319 20:29:57.009666 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0319 20:29:57.046461 55982 cri.go:89] found id: ""
I0319 20:29:57.046493 55982 logs.go:276] 0 containers: []
W0319 20:29:57.046504 55982 logs.go:278] No container was found matching "kube-proxy"
I0319 20:29:57.046511 55982 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0319 20:29:57.046579 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0319 20:29:57.084091 55982 cri.go:89] found id: ""
I0319 20:29:57.084118 55982 logs.go:276] 0 containers: []
W0319 20:29:57.084129 55982 logs.go:278] No container was found matching "kube-controller-manager"
I0319 20:29:57.084136 55982 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0319 20:29:57.084195 55982 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0319 20:29:57.122334 55982 cri.go:89] found id: ""
I0319 20:29:57.122365 55982 logs.go:276] 0 containers: []
W0319 20:29:57.122377 55982 logs.go:278] No container was found matching "kindnet"
I0319 20:29:57.122387 55982 logs.go:123] Gathering logs for container status ...
I0319 20:29:57.122402 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0319 20:29:57.165894 55982 logs.go:123] Gathering logs for kubelet ...
I0319 20:29:57.165929 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0319 20:29:57.231122 55982 logs.go:123] Gathering logs for dmesg ...
I0319 20:29:57.231157 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0319 20:29:57.247995 55982 logs.go:123] Gathering logs for describe nodes ...
I0319 20:29:57.248037 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0319 20:29:57.386567 55982 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0319 20:29:57.386598 55982 logs.go:123] Gathering logs for CRI-O ...
I0319 20:29:57.386614 55982 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
W0319 20:29:57.489187 55982 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0319 20:29:57.489237 55982 out.go:239] *
*
W0319 20:29:57.489297 55982 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0319 20:29:57.489356 55982 out.go:239] *
*
W0319 20:29:57.490139 55982 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0319 20:29:57.493459 55982 out.go:177]
W0319 20:29:57.494907 55982 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0319 20:29:57.494960 55982 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0319 20:29:57.494987 55982 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0319 20:29:57.496766 55982 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-159022 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-159022 -n old-k8s-version-159022
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-159022 -n old-k8s-version-159022: exit status 6 (256.332072ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0319 20:29:57.801509 58734 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-159022" does not appear in /home/jenkins/minikube-integration/18453-10028/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-159022" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (311.55s)