=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-986088 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-986088 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m43.790679053s)
-- stdout --
* [old-k8s-version-986088] minikube v1.33.0-beta.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=18485
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18485-1069254/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18485-1069254/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-986088" primary control-plane node in "old-k8s-version-986088" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0328 00:52:42.097217 1124641 out.go:291] Setting OutFile to fd 1 ...
I0328 00:52:42.097371 1124641 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 00:52:42.097384 1124641 out.go:304] Setting ErrFile to fd 2...
I0328 00:52:42.097390 1124641 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 00:52:42.097610 1124641 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18485-1069254/.minikube/bin
I0328 00:52:42.098253 1124641 out.go:298] Setting JSON to false
I0328 00:52:42.099533 1124641 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-9","uptime":30859,"bootTime":1711556303,"procs":302,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1054-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0328 00:52:42.099606 1124641 start.go:139] virtualization: kvm guest
I0328 00:52:42.101940 1124641 out.go:177] * [old-k8s-version-986088] minikube v1.33.0-beta.0 on Ubuntu 20.04 (kvm/amd64)
I0328 00:52:42.103395 1124641 notify.go:220] Checking for updates...
I0328 00:52:42.103416 1124641 out.go:177] - MINIKUBE_LOCATION=18485
I0328 00:52:42.104648 1124641 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0328 00:52:42.105963 1124641 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18485-1069254/kubeconfig
I0328 00:52:42.107429 1124641 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18485-1069254/.minikube
I0328 00:52:42.108960 1124641 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0328 00:52:42.110447 1124641 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0328 00:52:42.112657 1124641 config.go:182] Loaded profile config "bridge-443419": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0328 00:52:42.112833 1124641 config.go:182] Loaded profile config "flannel-443419": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.29.3
I0328 00:52:42.112949 1124641 config.go:182] Loaded profile config "kubernetes-upgrade-615158": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.0-beta.0
I0328 00:52:42.113110 1124641 driver.go:392] Setting default libvirt URI to qemu:///system
I0328 00:52:42.157704 1124641 out.go:177] * Using the kvm2 driver based on user configuration
I0328 00:52:42.159082 1124641 start.go:297] selected driver: kvm2
I0328 00:52:42.159106 1124641 start.go:901] validating driver "kvm2" against <nil>
I0328 00:52:42.159123 1124641 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0328 00:52:42.160207 1124641 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 00:52:42.160338 1124641 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/18485-1069254/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0328 00:52:42.176975 1124641 install.go:137] /home/jenkins/minikube-integration/18485-1069254/.minikube/bin/docker-machine-driver-kvm2 version is 1.33.0-beta.0
I0328 00:52:42.177037 1124641 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0328 00:52:42.177283 1124641 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0328 00:52:42.177375 1124641 cni.go:84] Creating CNI manager for ""
I0328 00:52:42.177395 1124641 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0328 00:52:42.177411 1124641 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0328 00:52:42.177488 1124641 start.go:340] cluster config:
{Name:old-k8s-version-986088 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-986088 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 00:52:42.177608 1124641 iso.go:125] acquiring lock: {Name:mk3da1fa7d63a581c817f327d86a827697458cb8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 00:52:42.180532 1124641 out.go:177] * Starting "old-k8s-version-986088" primary control-plane node in "old-k8s-version-986088" cluster
I0328 00:52:42.181838 1124641 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0328 00:52:42.181893 1124641 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0328 00:52:42.181901 1124641 cache.go:56] Caching tarball of preloaded images
I0328 00:52:42.182019 1124641 preload.go:173] Found /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0328 00:52:42.182036 1124641 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0328 00:52:42.182169 1124641 profile.go:142] Saving config to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/config.json ...
I0328 00:52:42.182193 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/config.json: {Name:mk2a2b625e7c87b41aece60741847db61cc5c6d0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:52:42.182370 1124641 start.go:360] acquireMachinesLock for old-k8s-version-986088: {Name:mk85e225431128bbd27ac7bb3815095957281902 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0328 00:52:50.307573 1124641 start.go:364] duration metric: took 8.125152123s to acquireMachinesLock for "old-k8s-version-986088"
I0328 00:52:50.307660 1124641 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-986088 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18485/minikube-v1.33.0-1711559712-18485-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-986088 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0328 00:52:50.307798 1124641 start.go:125] createHost starting for "" (driver="kvm2")
I0328 00:52:50.310866 1124641 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0328 00:52:50.311066 1124641 main.go:141] libmachine: Found binary path at /home/jenkins/minikube-integration/18485-1069254/.minikube/bin/docker-machine-driver-kvm2
I0328 00:52:50.311116 1124641 main.go:141] libmachine: Launching plugin server for driver kvm2
I0328 00:52:50.328919 1124641 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:38445
I0328 00:52:50.329482 1124641 main.go:141] libmachine: () Calling .GetVersion
I0328 00:52:50.330053 1124641 main.go:141] libmachine: Using API Version 1
I0328 00:52:50.330080 1124641 main.go:141] libmachine: () Calling .SetConfigRaw
I0328 00:52:50.330484 1124641 main.go:141] libmachine: () Calling .GetMachineName
I0328 00:52:50.330741 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetMachineName
I0328 00:52:50.330894 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:52:50.331093 1124641 start.go:159] libmachine.API.Create for "old-k8s-version-986088" (driver="kvm2")
I0328 00:52:50.331146 1124641 client.go:168] LocalClient.Create starting
I0328 00:52:50.331188 1124641 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem
I0328 00:52:50.331231 1124641 main.go:141] libmachine: Decoding PEM data...
I0328 00:52:50.331255 1124641 main.go:141] libmachine: Parsing certificate...
I0328 00:52:50.331327 1124641 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/cert.pem
I0328 00:52:50.331353 1124641 main.go:141] libmachine: Decoding PEM data...
I0328 00:52:50.331367 1124641 main.go:141] libmachine: Parsing certificate...
I0328 00:52:50.331393 1124641 main.go:141] libmachine: Running pre-create checks...
I0328 00:52:50.331406 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .PreCreateCheck
I0328 00:52:50.331828 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetConfigRaw
I0328 00:52:50.332301 1124641 main.go:141] libmachine: Creating machine...
I0328 00:52:50.332320 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .Create
I0328 00:52:50.332471 1124641 main.go:141] libmachine: (old-k8s-version-986088) Creating KVM machine...
I0328 00:52:50.333718 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found existing default KVM network
I0328 00:52:50.335024 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.334874 1124865 network.go:211] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName:virbr1 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:7c:ba:23} reservation:<nil>}
I0328 00:52:50.336007 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.335919 1124865 network.go:206] using free private subnet 192.168.50.0/24: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000282960}
I0328 00:52:50.336076 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | created network xml:
I0328 00:52:50.336105 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <network>
I0328 00:52:50.336136 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <name>mk-old-k8s-version-986088</name>
I0328 00:52:50.336143 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <dns enable='no'/>
I0328 00:52:50.336151 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG |
I0328 00:52:50.336168 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <ip address='192.168.50.1' netmask='255.255.255.0'>
I0328 00:52:50.336199 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <dhcp>
I0328 00:52:50.336236 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | <range start='192.168.50.2' end='192.168.50.253'/>
I0328 00:52:50.336250 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | </dhcp>
I0328 00:52:50.336259 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | </ip>
I0328 00:52:50.336270 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG |
I0328 00:52:50.336282 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | </network>
I0328 00:52:50.336295 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG |
I0328 00:52:50.341837 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | trying to create private KVM network mk-old-k8s-version-986088 192.168.50.0/24...
I0328 00:52:50.413785 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | private KVM network mk-old-k8s-version-986088 192.168.50.0/24 created
I0328 00:52:50.413815 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting up store path in /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088 ...
I0328 00:52:50.413831 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.413745 1124865 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/18485-1069254/.minikube
I0328 00:52:50.413850 1124641 main.go:141] libmachine: (old-k8s-version-986088) Building disk image from file:///home/jenkins/minikube-integration/18485-1069254/.minikube/cache/iso/amd64/minikube-v1.33.0-1711559712-18485-amd64.iso
I0328 00:52:50.413917 1124641 main.go:141] libmachine: (old-k8s-version-986088) Downloading /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/18485-1069254/.minikube/cache/iso/amd64/minikube-v1.33.0-1711559712-18485-amd64.iso...
I0328 00:52:50.671525 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.671386 1124865 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa...
I0328 00:52:50.772019 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.771866 1124865 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/old-k8s-version-986088.rawdisk...
I0328 00:52:50.772065 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Writing magic tar header
I0328 00:52:50.772117 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Writing SSH key tar header
I0328 00:52:50.772149 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:50.772022 1124865 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088 ...
I0328 00:52:50.772179 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088
I0328 00:52:50.772196 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines
I0328 00:52:50.772214 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18485-1069254/.minikube
I0328 00:52:50.772234 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18485-1069254
I0328 00:52:50.772246 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0328 00:52:50.772261 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088 (perms=drwx------)
I0328 00:52:50.772266 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home/jenkins
I0328 00:52:50.772286 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Checking permissions on dir: /home
I0328 00:52:50.772294 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Skipping /home - not owner
I0328 00:52:50.772311 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins/minikube-integration/18485-1069254/.minikube/machines (perms=drwxr-xr-x)
I0328 00:52:50.772331 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins/minikube-integration/18485-1069254/.minikube (perms=drwxr-xr-x)
I0328 00:52:50.772349 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins/minikube-integration/18485-1069254 (perms=drwxrwxr-x)
I0328 00:52:50.772363 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0328 00:52:50.772381 1124641 main.go:141] libmachine: (old-k8s-version-986088) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0328 00:52:50.772405 1124641 main.go:141] libmachine: (old-k8s-version-986088) Creating domain...
I0328 00:52:50.773602 1124641 main.go:141] libmachine: (old-k8s-version-986088) define libvirt domain using xml:
I0328 00:52:50.773629 1124641 main.go:141] libmachine: (old-k8s-version-986088) <domain type='kvm'>
I0328 00:52:50.773640 1124641 main.go:141] libmachine: (old-k8s-version-986088) <name>old-k8s-version-986088</name>
I0328 00:52:50.773647 1124641 main.go:141] libmachine: (old-k8s-version-986088) <memory unit='MiB'>2200</memory>
I0328 00:52:50.773656 1124641 main.go:141] libmachine: (old-k8s-version-986088) <vcpu>2</vcpu>
I0328 00:52:50.773666 1124641 main.go:141] libmachine: (old-k8s-version-986088) <features>
I0328 00:52:50.773675 1124641 main.go:141] libmachine: (old-k8s-version-986088) <acpi/>
I0328 00:52:50.773695 1124641 main.go:141] libmachine: (old-k8s-version-986088) <apic/>
I0328 00:52:50.773743 1124641 main.go:141] libmachine: (old-k8s-version-986088) <pae/>
I0328 00:52:50.773770 1124641 main.go:141] libmachine: (old-k8s-version-986088)
I0328 00:52:50.773782 1124641 main.go:141] libmachine: (old-k8s-version-986088) </features>
I0328 00:52:50.773885 1124641 main.go:141] libmachine: (old-k8s-version-986088) <cpu mode='host-passthrough'>
I0328 00:52:50.773914 1124641 main.go:141] libmachine: (old-k8s-version-986088)
I0328 00:52:50.773927 1124641 main.go:141] libmachine: (old-k8s-version-986088) </cpu>
I0328 00:52:50.773935 1124641 main.go:141] libmachine: (old-k8s-version-986088) <os>
I0328 00:52:50.773948 1124641 main.go:141] libmachine: (old-k8s-version-986088) <type>hvm</type>
I0328 00:52:50.773958 1124641 main.go:141] libmachine: (old-k8s-version-986088) <boot dev='cdrom'/>
I0328 00:52:50.773968 1124641 main.go:141] libmachine: (old-k8s-version-986088) <boot dev='hd'/>
I0328 00:52:50.773979 1124641 main.go:141] libmachine: (old-k8s-version-986088) <bootmenu enable='no'/>
I0328 00:52:50.773992 1124641 main.go:141] libmachine: (old-k8s-version-986088) </os>
I0328 00:52:50.774002 1124641 main.go:141] libmachine: (old-k8s-version-986088) <devices>
I0328 00:52:50.774010 1124641 main.go:141] libmachine: (old-k8s-version-986088) <disk type='file' device='cdrom'>
I0328 00:52:50.774025 1124641 main.go:141] libmachine: (old-k8s-version-986088) <source file='/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/boot2docker.iso'/>
I0328 00:52:50.774042 1124641 main.go:141] libmachine: (old-k8s-version-986088) <target dev='hdc' bus='scsi'/>
I0328 00:52:50.774055 1124641 main.go:141] libmachine: (old-k8s-version-986088) <readonly/>
I0328 00:52:50.774065 1124641 main.go:141] libmachine: (old-k8s-version-986088) </disk>
I0328 00:52:50.774084 1124641 main.go:141] libmachine: (old-k8s-version-986088) <disk type='file' device='disk'>
I0328 00:52:50.774106 1124641 main.go:141] libmachine: (old-k8s-version-986088) <driver name='qemu' type='raw' cache='default' io='threads' />
I0328 00:52:50.774140 1124641 main.go:141] libmachine: (old-k8s-version-986088) <source file='/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/old-k8s-version-986088.rawdisk'/>
I0328 00:52:50.774154 1124641 main.go:141] libmachine: (old-k8s-version-986088) <target dev='hda' bus='virtio'/>
I0328 00:52:50.774169 1124641 main.go:141] libmachine: (old-k8s-version-986088) </disk>
I0328 00:52:50.774181 1124641 main.go:141] libmachine: (old-k8s-version-986088) <interface type='network'>
I0328 00:52:50.774193 1124641 main.go:141] libmachine: (old-k8s-version-986088) <source network='mk-old-k8s-version-986088'/>
I0328 00:52:50.774205 1124641 main.go:141] libmachine: (old-k8s-version-986088) <model type='virtio'/>
I0328 00:52:50.774213 1124641 main.go:141] libmachine: (old-k8s-version-986088) </interface>
I0328 00:52:50.774225 1124641 main.go:141] libmachine: (old-k8s-version-986088) <interface type='network'>
I0328 00:52:50.774249 1124641 main.go:141] libmachine: (old-k8s-version-986088) <source network='default'/>
I0328 00:52:50.774264 1124641 main.go:141] libmachine: (old-k8s-version-986088) <model type='virtio'/>
I0328 00:52:50.774272 1124641 main.go:141] libmachine: (old-k8s-version-986088) </interface>
I0328 00:52:50.774284 1124641 main.go:141] libmachine: (old-k8s-version-986088) <serial type='pty'>
I0328 00:52:50.774292 1124641 main.go:141] libmachine: (old-k8s-version-986088) <target port='0'/>
I0328 00:52:50.774302 1124641 main.go:141] libmachine: (old-k8s-version-986088) </serial>
I0328 00:52:50.774314 1124641 main.go:141] libmachine: (old-k8s-version-986088) <console type='pty'>
I0328 00:52:50.774358 1124641 main.go:141] libmachine: (old-k8s-version-986088) <target type='serial' port='0'/>
I0328 00:52:50.774386 1124641 main.go:141] libmachine: (old-k8s-version-986088) </console>
I0328 00:52:50.774398 1124641 main.go:141] libmachine: (old-k8s-version-986088) <rng model='virtio'>
I0328 00:52:50.774409 1124641 main.go:141] libmachine: (old-k8s-version-986088) <backend model='random'>/dev/random</backend>
I0328 00:52:50.774419 1124641 main.go:141] libmachine: (old-k8s-version-986088) </rng>
I0328 00:52:50.774431 1124641 main.go:141] libmachine: (old-k8s-version-986088)
I0328 00:52:50.774443 1124641 main.go:141] libmachine: (old-k8s-version-986088)
I0328 00:52:50.774454 1124641 main.go:141] libmachine: (old-k8s-version-986088) </devices>
I0328 00:52:50.774465 1124641 main.go:141] libmachine: (old-k8s-version-986088) </domain>
I0328 00:52:50.774476 1124641 main.go:141] libmachine: (old-k8s-version-986088)
I0328 00:52:50.778726 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:2f:c5:ad in network default
I0328 00:52:50.779420 1124641 main.go:141] libmachine: (old-k8s-version-986088) Ensuring networks are active...
I0328 00:52:50.779449 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:50.780294 1124641 main.go:141] libmachine: (old-k8s-version-986088) Ensuring network default is active
I0328 00:52:50.780633 1124641 main.go:141] libmachine: (old-k8s-version-986088) Ensuring network mk-old-k8s-version-986088 is active
I0328 00:52:50.781222 1124641 main.go:141] libmachine: (old-k8s-version-986088) Getting domain xml...
I0328 00:52:50.781949 1124641 main.go:141] libmachine: (old-k8s-version-986088) Creating domain...
I0328 00:52:52.059242 1124641 main.go:141] libmachine: (old-k8s-version-986088) Waiting to get IP...
I0328 00:52:52.060289 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:52.060816 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:52.060912 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:52.060811 1124865 retry.go:31] will retry after 266.370059ms: waiting for machine to come up
I0328 00:52:52.329334 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:52.330053 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:52.330082 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:52.329995 1124865 retry.go:31] will retry after 350.193758ms: waiting for machine to come up
I0328 00:52:52.681577 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:52.682253 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:52.682286 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:52.682175 1124865 retry.go:31] will retry after 480.060306ms: waiting for machine to come up
I0328 00:52:53.163584 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:53.164246 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:53.164283 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:53.164185 1124865 retry.go:31] will retry after 374.158905ms: waiting for machine to come up
I0328 00:52:53.539746 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:53.540392 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:53.540418 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:53.540332 1124865 retry.go:31] will retry after 499.267861ms: waiting for machine to come up
I0328 00:52:54.042965 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:54.043617 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:54.043650 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:54.043517 1124865 retry.go:31] will retry after 927.777381ms: waiting for machine to come up
I0328 00:52:54.973519 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:54.974045 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:54.974076 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:54.973987 1124865 retry.go:31] will retry after 1.164717466s: waiting for machine to come up
I0328 00:52:56.140756 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:56.141300 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:56.141345 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:56.141228 1124865 retry.go:31] will retry after 1.460098825s: waiting for machine to come up
I0328 00:52:57.603286 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:57.603839 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:57.603864 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:57.603802 1124865 retry.go:31] will retry after 1.557195286s: waiting for machine to come up
I0328 00:52:59.163427 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:52:59.163919 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:52:59.163946 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:52:59.163876 1124865 retry.go:31] will retry after 2.233336392s: waiting for machine to come up
I0328 00:53:01.398539 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:01.399111 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:53:01.399145 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:53:01.399036 1124865 retry.go:31] will retry after 2.570219642s: waiting for machine to come up
I0328 00:53:03.971722 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:03.972214 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:53:03.972245 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:53:03.972156 1124865 retry.go:31] will retry after 3.012723138s: waiting for machine to come up
I0328 00:53:06.987500 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:06.988027 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:53:06.988057 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:53:06.987959 1124865 retry.go:31] will retry after 3.380584648s: waiting for machine to come up
I0328 00:53:10.370781 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:10.371192 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find current IP address of domain old-k8s-version-986088 in network mk-old-k8s-version-986088
I0328 00:53:10.371222 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | I0328 00:53:10.371162 1124865 retry.go:31] will retry after 4.445548056s: waiting for machine to come up
I0328 00:53:14.819490 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:14.820045 1124641 main.go:141] libmachine: (old-k8s-version-986088) Found IP for machine: 192.168.50.174
I0328 00:53:14.820068 1124641 main.go:141] libmachine: (old-k8s-version-986088) Reserving static IP address...
I0328 00:53:14.820097 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has current primary IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:14.820406 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-986088", mac: "52:54:00:f6:94:40", ip: "192.168.50.174"} in network mk-old-k8s-version-986088
I0328 00:53:14.960546 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Getting to WaitForSSH function...
I0328 00:53:14.960578 1124641 main.go:141] libmachine: (old-k8s-version-986088) Reserved static IP address: 192.168.50.174
I0328 00:53:14.960591 1124641 main.go:141] libmachine: (old-k8s-version-986088) Waiting for SSH to be available...
I0328 00:53:14.964222 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:14.964681 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088
I0328 00:53:14.964714 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | unable to find defined IP address of network mk-old-k8s-version-986088 interface with MAC address 52:54:00:f6:94:40
I0328 00:53:14.964814 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Using SSH client type: external
I0328 00:53:14.964850 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Using SSH private key: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa (-rw-------)
I0328 00:53:14.964889 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@ -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa -p 22] /usr/bin/ssh <nil>}
I0328 00:53:14.964904 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | About to run SSH command:
I0328 00:53:14.964919 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | exit 0
I0328 00:53:14.968892 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | SSH cmd err, output: exit status 255:
I0328 00:53:14.968924 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Error getting ssh command 'exit 0' : ssh command error:
I0328 00:53:14.968932 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | command : exit 0
I0328 00:53:14.968939 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | err : exit status 255
I0328 00:53:14.968950 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | output :
I0328 00:53:17.971367 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Getting to WaitForSSH function...
I0328 00:53:17.973946 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:17.974492 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:17.974524 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:17.974661 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Using SSH client type: external
I0328 00:53:17.974697 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Using SSH private key: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa (-rw-------)
I0328 00:53:17.974732 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.50.174 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa -p 22] /usr/bin/ssh <nil>}
I0328 00:53:17.974750 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | About to run SSH command:
I0328 00:53:17.974764 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | exit 0
I0328 00:53:18.102765 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | SSH cmd err, output: <nil>:
I0328 00:53:18.103047 1124641 main.go:141] libmachine: (old-k8s-version-986088) KVM machine creation complete!
I0328 00:53:18.103410 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetConfigRaw
I0328 00:53:18.104013 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:18.104233 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:18.104442 1124641 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0328 00:53:18.104462 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetState
I0328 00:53:18.106100 1124641 main.go:141] libmachine: Detecting operating system of created instance...
I0328 00:53:18.106121 1124641 main.go:141] libmachine: Waiting for SSH to be available...
I0328 00:53:18.106130 1124641 main.go:141] libmachine: Getting to WaitForSSH function...
I0328 00:53:18.106139 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.109195 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.109639 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.109668 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.109844 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.110046 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.110206 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.110345 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.110569 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:18.110815 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:18.110831 1124641 main.go:141] libmachine: About to run SSH command:
exit 0
I0328 00:53:18.217817 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0328 00:53:18.217842 1124641 main.go:141] libmachine: Detecting the provisioner...
I0328 00:53:18.217851 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.220643 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.220930 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.220967 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.221201 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.221420 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.221602 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.221752 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.221952 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:18.222157 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:18.222171 1124641 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0328 00:53:18.327498 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0328 00:53:18.327605 1124641 main.go:141] libmachine: found compatible host: buildroot
I0328 00:53:18.327615 1124641 main.go:141] libmachine: Provisioning with buildroot...
I0328 00:53:18.327633 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetMachineName
I0328 00:53:18.327935 1124641 buildroot.go:166] provisioning hostname "old-k8s-version-986088"
I0328 00:53:18.327972 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetMachineName
I0328 00:53:18.328180 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.331461 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.331850 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.331880 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.332089 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.332303 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.332478 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.332630 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.332818 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:18.333017 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:18.333031 1124641 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-986088 && echo "old-k8s-version-986088" | sudo tee /etc/hostname
I0328 00:53:18.454355 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-986088
I0328 00:53:18.454391 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.457180 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.457549 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.457584 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.457758 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.457947 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.458113 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.458269 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.458485 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:18.458652 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:18.458669 1124641 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-986088' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-986088/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-986088' | sudo tee -a /etc/hosts;
fi
fi
I0328 00:53:18.578051 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0328 00:53:18.578090 1124641 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/18485-1069254/.minikube CaCertPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18485-1069254/.minikube}
I0328 00:53:18.578124 1124641 buildroot.go:174] setting up certificates
I0328 00:53:18.578138 1124641 provision.go:84] configureAuth start
I0328 00:53:18.578149 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetMachineName
I0328 00:53:18.578509 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetIP
I0328 00:53:18.581751 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.582195 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.582225 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.582432 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.584922 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.585266 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.585300 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.585440 1124641 provision.go:143] copyHostCerts
I0328 00:53:18.585524 1124641 exec_runner.go:144] found /home/jenkins/minikube-integration/18485-1069254/.minikube/key.pem, removing ...
I0328 00:53:18.585539 1124641 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18485-1069254/.minikube/key.pem
I0328 00:53:18.585613 1124641 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18485-1069254/.minikube/key.pem (1679 bytes)
I0328 00:53:18.585731 1124641 exec_runner.go:144] found /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.pem, removing ...
I0328 00:53:18.585743 1124641 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.pem
I0328 00:53:18.585775 1124641 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.pem (1078 bytes)
I0328 00:53:18.585912 1124641 exec_runner.go:144] found /home/jenkins/minikube-integration/18485-1069254/.minikube/cert.pem, removing ...
I0328 00:53:18.585926 1124641 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18485-1069254/.minikube/cert.pem
I0328 00:53:18.585958 1124641 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18485-1069254/.minikube/cert.pem (1123 bytes)
I0328 00:53:18.586026 1124641 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-986088 san=[127.0.0.1 192.168.50.174 localhost minikube old-k8s-version-986088]
I0328 00:53:18.672045 1124641 provision.go:177] copyRemoteCerts
I0328 00:53:18.672112 1124641 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0328 00:53:18.672139 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.675064 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.675408 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.675433 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.675618 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.675841 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.676034 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.676200 1124641 sshutil.go:53] new ssh client: &{IP:192.168.50.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa Username:docker}
I0328 00:53:18.757147 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0328 00:53:18.783750 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0328 00:53:18.810869 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0328 00:53:18.837121 1124641 provision.go:87] duration metric: took 258.970139ms to configureAuth
I0328 00:53:18.837151 1124641 buildroot.go:189] setting minikube options for container-runtime
I0328 00:53:18.837363 1124641 config.go:182] Loaded profile config "old-k8s-version-986088": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0328 00:53:18.837457 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:18.840371 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.840751 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:18.840783 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:18.841033 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:18.841251 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.841389 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:18.841506 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:18.841647 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:18.841875 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:18.841897 1124641 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0328 00:53:19.119256 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0328 00:53:19.119291 1124641 main.go:141] libmachine: Checking connection to Docker...
I0328 00:53:19.119302 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetURL
I0328 00:53:19.120644 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | Using libvirt version 6000000
I0328 00:53:19.123208 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.123562 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.123592 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.123790 1124641 main.go:141] libmachine: Docker is up and running!
I0328 00:53:19.123807 1124641 main.go:141] libmachine: Reticulating splines...
I0328 00:53:19.123816 1124641 client.go:171] duration metric: took 28.792657152s to LocalClient.Create
I0328 00:53:19.123850 1124641 start.go:167] duration metric: took 28.792757608s to libmachine.API.Create "old-k8s-version-986088"
I0328 00:53:19.123864 1124641 start.go:293] postStartSetup for "old-k8s-version-986088" (driver="kvm2")
I0328 00:53:19.123880 1124641 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0328 00:53:19.123911 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:19.124226 1124641 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0328 00:53:19.124262 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:19.126673 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.127034 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.127060 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.127259 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:19.127444 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:19.127601 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:19.127763 1124641 sshutil.go:53] new ssh client: &{IP:192.168.50.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa Username:docker}
I0328 00:53:19.211077 1124641 ssh_runner.go:195] Run: cat /etc/os-release
I0328 00:53:19.215630 1124641 info.go:137] Remote host: Buildroot 2023.02.9
I0328 00:53:19.215654 1124641 filesync.go:126] Scanning /home/jenkins/minikube-integration/18485-1069254/.minikube/addons for local assets ...
I0328 00:53:19.215721 1124641 filesync.go:126] Scanning /home/jenkins/minikube-integration/18485-1069254/.minikube/files for local assets ...
I0328 00:53:19.215810 1124641 filesync.go:149] local asset: /home/jenkins/minikube-integration/18485-1069254/.minikube/files/etc/ssl/certs/10765222.pem -> 10765222.pem in /etc/ssl/certs
I0328 00:53:19.215903 1124641 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0328 00:53:19.226790 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/files/etc/ssl/certs/10765222.pem --> /etc/ssl/certs/10765222.pem (1708 bytes)
I0328 00:53:19.253165 1124641 start.go:296] duration metric: took 129.284021ms for postStartSetup
I0328 00:53:19.253232 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetConfigRaw
I0328 00:53:19.253866 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetIP
I0328 00:53:19.256690 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.257020 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.257052 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.257270 1124641 profile.go:142] Saving config to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/config.json ...
I0328 00:53:19.257510 1124641 start.go:128] duration metric: took 28.949699418s to createHost
I0328 00:53:19.257550 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:19.259943 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.260319 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.260345 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.260531 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:19.260778 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:19.260917 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:19.261046 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:19.261172 1124641 main.go:141] libmachine: Using SSH client type: native
I0328 00:53:19.261358 1124641 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d4a0] 0x830200 <nil> [] 0s} 192.168.50.174 22 <nil> <nil>}
I0328 00:53:19.261371 1124641 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0328 00:53:19.367406 1124641 main.go:141] libmachine: SSH cmd err, output: <nil>: 1711587199.354010792
I0328 00:53:19.367435 1124641 fix.go:216] guest clock: 1711587199.354010792
I0328 00:53:19.367444 1124641 fix.go:229] Guest: 2024-03-28 00:53:19.354010792 +0000 UTC Remote: 2024-03-28 00:53:19.257524913 +0000 UTC m=+37.211927852 (delta=96.485879ms)
I0328 00:53:19.367471 1124641 fix.go:200] guest clock delta is within tolerance: 96.485879ms
I0328 00:53:19.367483 1124641 start.go:83] releasing machines lock for "old-k8s-version-986088", held for 29.059861742s
I0328 00:53:19.367518 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:19.367827 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetIP
I0328 00:53:19.370638 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.370975 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.371011 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.371239 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:19.371832 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:19.372032 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .DriverName
I0328 00:53:19.372130 1124641 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0328 00:53:19.372187 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:19.372299 1124641 ssh_runner.go:195] Run: cat /version.json
I0328 00:53:19.372328 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHHostname
I0328 00:53:19.374894 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.375245 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.375282 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.375303 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.375417 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:19.375603 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:19.375758 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:19.375760 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:19.375804 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:19.375941 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHPort
I0328 00:53:19.376072 1124641 sshutil.go:53] new ssh client: &{IP:192.168.50.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa Username:docker}
I0328 00:53:19.376137 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHKeyPath
I0328 00:53:19.376290 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetSSHUsername
I0328 00:53:19.376466 1124641 sshutil.go:53] new ssh client: &{IP:192.168.50.174 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18485-1069254/.minikube/machines/old-k8s-version-986088/id_rsa Username:docker}
I0328 00:53:19.452085 1124641 ssh_runner.go:195] Run: systemctl --version
I0328 00:53:19.489905 1124641 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0328 00:53:19.659124 1124641 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0328 00:53:19.666301 1124641 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0328 00:53:19.666367 1124641 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0328 00:53:19.683591 1124641 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0328 00:53:19.683625 1124641 start.go:494] detecting cgroup driver to use...
I0328 00:53:19.683712 1124641 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0328 00:53:19.702361 1124641 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0328 00:53:19.717428 1124641 docker.go:217] disabling cri-docker service (if available) ...
I0328 00:53:19.717515 1124641 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0328 00:53:19.734328 1124641 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0328 00:53:19.752378 1124641 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0328 00:53:19.886973 1124641 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0328 00:53:20.052331 1124641 docker.go:233] disabling docker service ...
I0328 00:53:20.052405 1124641 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0328 00:53:20.070362 1124641 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0328 00:53:20.085054 1124641 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0328 00:53:20.231327 1124641 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0328 00:53:20.353342 1124641 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0328 00:53:20.368767 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0328 00:53:20.391111 1124641 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0328 00:53:20.391185 1124641 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 00:53:20.403437 1124641 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0328 00:53:20.403526 1124641 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 00:53:20.416877 1124641 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0328 00:53:20.428550 1124641 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0328 00:53:20.439998 1124641 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0328 00:53:20.451361 1124641 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0328 00:53:20.461989 1124641 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0328 00:53:20.462061 1124641 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0328 00:53:20.476497 1124641 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0328 00:53:20.486201 1124641 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 00:53:20.607359 1124641 ssh_runner.go:195] Run: sudo systemctl restart crio
I0328 00:53:20.757311 1124641 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0328 00:53:20.757402 1124641 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0328 00:53:20.763002 1124641 start.go:562] Will wait 60s for crictl version
I0328 00:53:20.763089 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:20.767296 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0328 00:53:20.805442 1124641 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0328 00:53:20.805530 1124641 ssh_runner.go:195] Run: crio --version
I0328 00:53:20.837325 1124641 ssh_runner.go:195] Run: crio --version
I0328 00:53:20.878999 1124641 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0328 00:53:20.880558 1124641 main.go:141] libmachine: (old-k8s-version-986088) Calling .GetIP
I0328 00:53:20.884024 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:20.884515 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f6:94:40", ip: ""} in network mk-old-k8s-version-986088: {Iface:virbr2 ExpiryTime:2024-03-28 01:53:07 +0000 UTC Type:0 Mac:52:54:00:f6:94:40 Iaid: IPaddr:192.168.50.174 Prefix:24 Hostname:old-k8s-version-986088 Clientid:01:52:54:00:f6:94:40}
I0328 00:53:20.884547 1124641 main.go:141] libmachine: (old-k8s-version-986088) DBG | domain old-k8s-version-986088 has defined IP address 192.168.50.174 and MAC address 52:54:00:f6:94:40 in network mk-old-k8s-version-986088
I0328 00:53:20.884776 1124641 ssh_runner.go:195] Run: grep 192.168.50.1 host.minikube.internal$ /etc/hosts
I0328 00:53:20.890410 1124641 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.50.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 00:53:20.905292 1124641 kubeadm.go:877] updating cluster {Name:old-k8s-version-986088 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18485/minikube-v1.33.0-1711559712-18485-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-986088 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.174 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] M
ountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0328 00:53:20.905411 1124641 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0328 00:53:20.905461 1124641 ssh_runner.go:195] Run: sudo crictl images --output json
I0328 00:53:20.945202 1124641 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0328 00:53:20.945292 1124641 ssh_runner.go:195] Run: which lz4
I0328 00:53:20.950541 1124641 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0328 00:53:20.955787 1124641 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0328 00:53:20.955825 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0328 00:53:23.176266 1124641 crio.go:462] duration metric: took 2.22577854s to copy over tarball
I0328 00:53:23.176359 1124641 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0328 00:53:26.189942 1124641 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (3.013548607s)
I0328 00:53:26.189979 1124641 crio.go:469] duration metric: took 3.01367539s to extract the tarball
I0328 00:53:26.189991 1124641 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0328 00:53:26.240599 1124641 ssh_runner.go:195] Run: sudo crictl images --output json
I0328 00:53:26.293594 1124641 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0328 00:53:26.293637 1124641 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0328 00:53:26.293692 1124641 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0328 00:53:26.293716 1124641 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0328 00:53:26.293807 1124641 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0328 00:53:26.293825 1124641 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0328 00:53:26.293971 1124641 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0328 00:53:26.293993 1124641 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0328 00:53:26.294009 1124641 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0328 00:53:26.294077 1124641 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0328 00:53:26.295901 1124641 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0328 00:53:26.295935 1124641 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0328 00:53:26.295932 1124641 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0328 00:53:26.295938 1124641 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0328 00:53:26.295956 1124641 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0328 00:53:26.295979 1124641 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0328 00:53:26.295900 1124641 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0328 00:53:26.295906 1124641 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0328 00:53:26.482866 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0328 00:53:26.514936 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0328 00:53:26.518806 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0328 00:53:26.519786 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0328 00:53:26.536745 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0328 00:53:26.537197 1124641 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0328 00:53:26.537244 1124641 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0328 00:53:26.537287 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.551945 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0328 00:53:26.555802 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0328 00:53:26.637065 1124641 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0328 00:53:26.637126 1124641 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0328 00:53:26.637187 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.660649 1124641 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0328 00:53:26.660712 1124641 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0328 00:53:26.660770 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.677564 1124641 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0328 00:53:26.677609 1124641 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0328 00:53:26.677662 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.702153 1124641 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0328 00:53:26.702202 1124641 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0328 00:53:26.702219 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0328 00:53:26.702268 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.702344 1124641 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0328 00:53:26.702393 1124641 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0328 00:53:26.702430 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.707238 1124641 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0328 00:53:26.707292 1124641 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0328 00:53:26.707322 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0328 00:53:26.707331 1124641 ssh_runner.go:195] Run: which crictl
I0328 00:53:26.707340 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0328 00:53:26.707349 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0328 00:53:26.717118 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0328 00:53:26.717193 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0328 00:53:26.790132 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0328 00:53:26.860423 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0328 00:53:26.860517 1124641 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0328 00:53:26.860550 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0328 00:53:26.860641 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0328 00:53:26.860710 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0328 00:53:26.864195 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0328 00:53:26.898469 1124641 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0328 00:53:27.176096 1124641 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0328 00:53:27.365610 1124641 cache_images.go:92] duration metric: took 1.071949881s to LoadCachedImages
W0328 00:53:27.365720 1124641 out.go:239] X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18485-1069254/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0: no such file or directory
I0328 00:53:27.365738 1124641 kubeadm.go:928] updating node { 192.168.50.174 8443 v1.20.0 crio true true} ...
I0328 00:53:27.365933 1124641 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-986088 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.50.174
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-986088 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0328 00:53:27.366025 1124641 ssh_runner.go:195] Run: crio config
I0328 00:53:27.418740 1124641 cni.go:84] Creating CNI manager for ""
I0328 00:53:27.418764 1124641 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0328 00:53:27.418773 1124641 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0328 00:53:27.418793 1124641 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.50.174 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-986088 NodeName:old-k8s-version-986088 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.50.174"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.50.174 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt St
aticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0328 00:53:27.418951 1124641 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.50.174
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-986088"
kubeletExtraArgs:
node-ip: 192.168.50.174
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.50.174"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0328 00:53:27.419018 1124641 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0328 00:53:27.430170 1124641 binaries.go:44] Found k8s binaries, skipping transfer
I0328 00:53:27.430279 1124641 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0328 00:53:27.441641 1124641 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (430 bytes)
I0328 00:53:27.465265 1124641 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0328 00:53:27.484812 1124641 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2123 bytes)
I0328 00:53:27.506907 1124641 ssh_runner.go:195] Run: grep 192.168.50.174 control-plane.minikube.internal$ /etc/hosts
I0328 00:53:27.512398 1124641 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.50.174 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 00:53:27.526372 1124641 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 00:53:27.654673 1124641 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0328 00:53:27.673162 1124641 certs.go:68] Setting up /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088 for IP: 192.168.50.174
I0328 00:53:27.673191 1124641 certs.go:194] generating shared ca certs ...
I0328 00:53:27.673214 1124641 certs.go:226] acquiring lock for ca certs: {Name:mkf4dec8f33bbf51de6ed3aabdf175c7fd744ef9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:27.673403 1124641 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.key
I0328 00:53:27.673457 1124641 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/proxy-client-ca.key
I0328 00:53:27.673469 1124641 certs.go:256] generating profile certs ...
I0328 00:53:27.673549 1124641 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.key
I0328 00:53:27.673591 1124641 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.crt with IP's: []
I0328 00:53:27.780901 1124641 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.crt ...
I0328 00:53:27.780943 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.crt: {Name:mk23aef8836c87e1b89d094f95ca1a5396294bba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:27.781174 1124641 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.key ...
I0328 00:53:27.781195 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/client.key: {Name:mkdafb5c59414df473b61728e39314ba035e5a28 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:27.781326 1124641 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key.b88fbc7e
I0328 00:53:27.781346 1124641 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt.b88fbc7e with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.50.174]
I0328 00:53:27.985113 1124641 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt.b88fbc7e ...
I0328 00:53:27.985147 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt.b88fbc7e: {Name:mk27005d9ef1375492fa98ecc68e2cc8ffd572d8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:27.985336 1124641 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key.b88fbc7e ...
I0328 00:53:27.985354 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key.b88fbc7e: {Name:mk09f92ef171af55c74a510d613a909e85a3e609 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:27.985469 1124641 certs.go:381] copying /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt.b88fbc7e -> /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt
I0328 00:53:27.985579 1124641 certs.go:385] copying /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key.b88fbc7e -> /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key
I0328 00:53:27.985650 1124641 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.key
I0328 00:53:27.985670 1124641 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.crt with IP's: []
I0328 00:53:28.107873 1124641 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.crt ...
I0328 00:53:28.107912 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.crt: {Name:mk0142c92145e3f5a0474994f829f182c527278c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:28.108089 1124641 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.key ...
I0328 00:53:28.108103 1124641 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.key: {Name:mkb316b567044b46087cf46fc69115e61af8813e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 00:53:28.108283 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/1076522.pem (1338 bytes)
W0328 00:53:28.108321 1124641 certs.go:480] ignoring /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/1076522_empty.pem, impossibly tiny 0 bytes
I0328 00:53:28.108332 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca-key.pem (1675 bytes)
I0328 00:53:28.108353 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/ca.pem (1078 bytes)
I0328 00:53:28.108375 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/cert.pem (1123 bytes)
I0328 00:53:28.108397 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/key.pem (1679 bytes)
I0328 00:53:28.108432 1124641 certs.go:484] found cert: /home/jenkins/minikube-integration/18485-1069254/.minikube/files/etc/ssl/certs/10765222.pem (1708 bytes)
I0328 00:53:28.109165 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0328 00:53:28.137925 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0328 00:53:28.167914 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0328 00:53:28.196566 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0328 00:53:28.225295 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0328 00:53:28.253854 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0328 00:53:28.279219 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0328 00:53:28.307921 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/profiles/old-k8s-version-986088/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0328 00:53:28.352262 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/certs/1076522.pem --> /usr/share/ca-certificates/1076522.pem (1338 bytes)
I0328 00:53:28.378773 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/files/etc/ssl/certs/10765222.pem --> /usr/share/ca-certificates/10765222.pem (1708 bytes)
I0328 00:53:28.405258 1124641 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18485-1069254/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0328 00:53:28.431779 1124641 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0328 00:53:28.451798 1124641 ssh_runner.go:195] Run: openssl version
I0328 00:53:28.458572 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1076522.pem && ln -fs /usr/share/ca-certificates/1076522.pem /etc/ssl/certs/1076522.pem"
I0328 00:53:28.471035 1124641 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1076522.pem
I0328 00:53:28.476199 1124641 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 27 23:42 /usr/share/ca-certificates/1076522.pem
I0328 00:53:28.476288 1124641 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1076522.pem
I0328 00:53:28.483067 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1076522.pem /etc/ssl/certs/51391683.0"
I0328 00:53:28.495812 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/10765222.pem && ln -fs /usr/share/ca-certificates/10765222.pem /etc/ssl/certs/10765222.pem"
I0328 00:53:28.511696 1124641 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/10765222.pem
I0328 00:53:28.522353 1124641 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 27 23:42 /usr/share/ca-certificates/10765222.pem
I0328 00:53:28.522434 1124641 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/10765222.pem
I0328 00:53:28.531737 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/10765222.pem /etc/ssl/certs/3ec20f2e.0"
I0328 00:53:28.549095 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0328 00:53:28.576030 1124641 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0328 00:53:28.594892 1124641 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 27 23:34 /usr/share/ca-certificates/minikubeCA.pem
I0328 00:53:28.594973 1124641 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0328 00:53:28.602737 1124641 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0328 00:53:28.616182 1124641 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0328 00:53:28.622016 1124641 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0328 00:53:28.622085 1124641 kubeadm.go:391] StartCluster: {Name:old-k8s-version-986088 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18485/minikube-v1.33.0-1711559712-18485-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-986088 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.50.174 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Moun
tPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 00:53:28.622189 1124641 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0328 00:53:28.622269 1124641 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0328 00:53:28.670277 1124641 cri.go:89] found id: ""
I0328 00:53:28.670363 1124641 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0328 00:53:28.682489 1124641 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0328 00:53:28.694029 1124641 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0328 00:53:28.705080 1124641 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0328 00:53:28.705109 1124641 kubeadm.go:156] found existing configuration files:
I0328 00:53:28.705173 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0328 00:53:28.715562 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0328 00:53:28.715644 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0328 00:53:28.726381 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0328 00:53:28.737489 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0328 00:53:28.737557 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0328 00:53:28.748639 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0328 00:53:28.759045 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0328 00:53:28.759121 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0328 00:53:28.771358 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0328 00:53:28.782604 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0328 00:53:28.782687 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0328 00:53:28.793067 1124641 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0328 00:53:29.086610 1124641 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0328 00:55:26.950870 1124641 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0328 00:55:26.951082 1124641 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0328 00:55:26.952464 1124641 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0328 00:55:26.952560 1124641 kubeadm.go:309] [preflight] Running pre-flight checks
I0328 00:55:26.952706 1124641 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0328 00:55:26.952903 1124641 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0328 00:55:26.953104 1124641 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0328 00:55:26.953241 1124641 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0328 00:55:26.957650 1124641 out.go:204] - Generating certificates and keys ...
I0328 00:55:26.957765 1124641 kubeadm.go:309] [certs] Using existing ca certificate authority
I0328 00:55:26.957868 1124641 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0328 00:55:26.957967 1124641 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0328 00:55:26.958066 1124641 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0328 00:55:26.958155 1124641 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0328 00:55:26.958268 1124641 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0328 00:55:26.958344 1124641 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0328 00:55:26.958481 1124641 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
I0328 00:55:26.958532 1124641 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0328 00:55:26.958656 1124641 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
I0328 00:55:26.958743 1124641 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0328 00:55:26.958828 1124641 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0328 00:55:26.958895 1124641 kubeadm.go:309] [certs] Generating "sa" key and public key
I0328 00:55:26.958972 1124641 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0328 00:55:26.959044 1124641 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0328 00:55:26.959124 1124641 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0328 00:55:26.959228 1124641 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0328 00:55:26.959308 1124641 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0328 00:55:26.959464 1124641 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0328 00:55:26.959583 1124641 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0328 00:55:26.959637 1124641 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0328 00:55:26.959697 1124641 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0328 00:55:26.961427 1124641 out.go:204] - Booting up control plane ...
I0328 00:55:26.961543 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0328 00:55:26.961642 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0328 00:55:26.961723 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0328 00:55:26.961836 1124641 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0328 00:55:26.962028 1124641 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0328 00:55:26.962116 1124641 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0328 00:55:26.962252 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:55:26.962525 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:55:26.962642 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:55:26.962861 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:55:26.962960 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:55:26.963232 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:55:26.963305 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:55:26.963461 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:55:26.963529 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:55:26.963686 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:55:26.963698 1124641 kubeadm.go:309]
I0328 00:55:26.963767 1124641 kubeadm.go:309] Unfortunately, an error has occurred:
I0328 00:55:26.963824 1124641 kubeadm.go:309] timed out waiting for the condition
I0328 00:55:26.963837 1124641 kubeadm.go:309]
I0328 00:55:26.963883 1124641 kubeadm.go:309] This error is likely caused by:
I0328 00:55:26.963925 1124641 kubeadm.go:309] - The kubelet is not running
I0328 00:55:26.964048 1124641 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0328 00:55:26.964061 1124641 kubeadm.go:309]
I0328 00:55:26.964188 1124641 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0328 00:55:26.964239 1124641 kubeadm.go:309] - 'systemctl status kubelet'
I0328 00:55:26.964281 1124641 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0328 00:55:26.964290 1124641 kubeadm.go:309]
I0328 00:55:26.964448 1124641 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0328 00:55:26.964577 1124641 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0328 00:55:26.964587 1124641 kubeadm.go:309]
I0328 00:55:26.964721 1124641 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0328 00:55:26.964847 1124641 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0328 00:55:26.964959 1124641 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0328 00:55:26.965068 1124641 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0328 00:55:26.965104 1124641 kubeadm.go:309]
W0328 00:55:26.965242 1124641 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-986088] and IPs [192.168.50.174 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0328 00:55:26.965297 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0328 00:55:28.545593 1124641 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force": (1.580268305s)
I0328 00:55:28.545677 1124641 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0328 00:55:28.563400 1124641 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0328 00:55:28.576173 1124641 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0328 00:55:28.576203 1124641 kubeadm.go:156] found existing configuration files:
I0328 00:55:28.576271 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0328 00:55:28.588411 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0328 00:55:28.588490 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0328 00:55:28.601004 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0328 00:55:28.613071 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0328 00:55:28.613143 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0328 00:55:28.624733 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0328 00:55:28.637281 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0328 00:55:28.637348 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0328 00:55:28.652505 1124641 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0328 00:55:28.668012 1124641 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0328 00:55:28.668082 1124641 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0328 00:55:28.681019 1124641 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0328 00:55:28.921104 1124641 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0328 00:57:25.174807 1124641 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0328 00:57:25.174935 1124641 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0328 00:57:25.176648 1124641 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0328 00:57:25.176761 1124641 kubeadm.go:309] [preflight] Running pre-flight checks
I0328 00:57:25.176899 1124641 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0328 00:57:25.177033 1124641 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0328 00:57:25.177159 1124641 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0328 00:57:25.177237 1124641 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0328 00:57:25.178691 1124641 out.go:204] - Generating certificates and keys ...
I0328 00:57:25.178782 1124641 kubeadm.go:309] [certs] Using existing ca certificate authority
I0328 00:57:25.178838 1124641 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0328 00:57:25.178904 1124641 kubeadm.go:309] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0328 00:57:25.179002 1124641 kubeadm.go:309] [certs] Using existing front-proxy-ca certificate authority
I0328 00:57:25.179104 1124641 kubeadm.go:309] [certs] Using existing front-proxy-client certificate and key on disk
I0328 00:57:25.179150 1124641 kubeadm.go:309] [certs] Using existing etcd/ca certificate authority
I0328 00:57:25.179202 1124641 kubeadm.go:309] [certs] Using existing etcd/server certificate and key on disk
I0328 00:57:25.179254 1124641 kubeadm.go:309] [certs] Using existing etcd/peer certificate and key on disk
I0328 00:57:25.179315 1124641 kubeadm.go:309] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0328 00:57:25.179439 1124641 kubeadm.go:309] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0328 00:57:25.179506 1124641 kubeadm.go:309] [certs] Using the existing "sa" key
I0328 00:57:25.179589 1124641 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0328 00:57:25.179668 1124641 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0328 00:57:25.179756 1124641 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0328 00:57:25.179850 1124641 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0328 00:57:25.179936 1124641 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0328 00:57:25.180072 1124641 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0328 00:57:25.180207 1124641 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0328 00:57:25.180264 1124641 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0328 00:57:25.180351 1124641 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0328 00:57:25.181881 1124641 out.go:204] - Booting up control plane ...
I0328 00:57:25.181986 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0328 00:57:25.182082 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0328 00:57:25.182210 1124641 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0328 00:57:25.182347 1124641 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0328 00:57:25.182587 1124641 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0328 00:57:25.182664 1124641 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0328 00:57:25.182770 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:57:25.183041 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:57:25.183139 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:57:25.183389 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:57:25.183506 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:57:25.183765 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:57:25.183858 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:57:25.184114 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:57:25.184190 1124641 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0328 00:57:25.184416 1124641 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0328 00:57:25.184429 1124641 kubeadm.go:309]
I0328 00:57:25.184474 1124641 kubeadm.go:309] Unfortunately, an error has occurred:
I0328 00:57:25.184534 1124641 kubeadm.go:309] timed out waiting for the condition
I0328 00:57:25.184543 1124641 kubeadm.go:309]
I0328 00:57:25.184593 1124641 kubeadm.go:309] This error is likely caused by:
I0328 00:57:25.184639 1124641 kubeadm.go:309] - The kubelet is not running
I0328 00:57:25.184772 1124641 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0328 00:57:25.184781 1124641 kubeadm.go:309]
I0328 00:57:25.184908 1124641 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0328 00:57:25.184956 1124641 kubeadm.go:309] - 'systemctl status kubelet'
I0328 00:57:25.185004 1124641 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0328 00:57:25.185017 1124641 kubeadm.go:309]
I0328 00:57:25.185147 1124641 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0328 00:57:25.185227 1124641 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0328 00:57:25.185235 1124641 kubeadm.go:309]
I0328 00:57:25.185338 1124641 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0328 00:57:25.185432 1124641 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0328 00:57:25.185517 1124641 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0328 00:57:25.185620 1124641 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0328 00:57:25.185662 1124641 kubeadm.go:309]
I0328 00:57:25.185702 1124641 kubeadm.go:393] duration metric: took 3m56.563621212s to StartCluster
I0328 00:57:25.185750 1124641 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 00:57:25.185806 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 00:57:25.233019 1124641 cri.go:89] found id: ""
I0328 00:57:25.233059 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.233074 1124641 logs.go:278] No container was found matching "kube-apiserver"
I0328 00:57:25.233084 1124641 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 00:57:25.233155 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 00:57:25.272283 1124641 cri.go:89] found id: ""
I0328 00:57:25.272315 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.272327 1124641 logs.go:278] No container was found matching "etcd"
I0328 00:57:25.272337 1124641 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 00:57:25.272398 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 00:57:25.309054 1124641 cri.go:89] found id: ""
I0328 00:57:25.309088 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.309097 1124641 logs.go:278] No container was found matching "coredns"
I0328 00:57:25.309104 1124641 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 00:57:25.309158 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 00:57:25.353057 1124641 cri.go:89] found id: ""
I0328 00:57:25.353094 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.353106 1124641 logs.go:278] No container was found matching "kube-scheduler"
I0328 00:57:25.353114 1124641 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 00:57:25.353192 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 00:57:25.391719 1124641 cri.go:89] found id: ""
I0328 00:57:25.391761 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.391774 1124641 logs.go:278] No container was found matching "kube-proxy"
I0328 00:57:25.391783 1124641 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 00:57:25.391844 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 00:57:25.431184 1124641 cri.go:89] found id: ""
I0328 00:57:25.431220 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.431232 1124641 logs.go:278] No container was found matching "kube-controller-manager"
I0328 00:57:25.431241 1124641 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 00:57:25.431312 1124641 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 00:57:25.470041 1124641 cri.go:89] found id: ""
I0328 00:57:25.470078 1124641 logs.go:276] 0 containers: []
W0328 00:57:25.470090 1124641 logs.go:278] No container was found matching "kindnet"
I0328 00:57:25.470105 1124641 logs.go:123] Gathering logs for describe nodes ...
I0328 00:57:25.470123 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0328 00:57:25.599003 1124641 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0328 00:57:25.599037 1124641 logs.go:123] Gathering logs for CRI-O ...
I0328 00:57:25.599058 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 00:57:25.696067 1124641 logs.go:123] Gathering logs for container status ...
I0328 00:57:25.696119 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 00:57:25.745831 1124641 logs.go:123] Gathering logs for kubelet ...
I0328 00:57:25.745873 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0328 00:57:25.798645 1124641 logs.go:123] Gathering logs for dmesg ...
I0328 00:57:25.798684 1124641 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
W0328 00:57:25.817016 1124641 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0328 00:57:25.817083 1124641 out.go:239] *
*
W0328 00:57:25.817163 1124641 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0328 00:57:25.817199 1124641 out.go:239] *
*
W0328 00:57:25.818454 1124641 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0328 00:57:25.822102 1124641 out.go:177]
W0328 00:57:25.823380 1124641 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0328 00:57:25.823443 1124641 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0328 00:57:25.823482 1124641 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0328 00:57:25.825011 1124641 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-986088 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-986088 -n old-k8s-version-986088
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-986088 -n old-k8s-version-986088: exit status 6 (265.974047ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0328 00:57:26.134895 1130531 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-986088" does not appear in /home/jenkins/minikube-integration/18485-1069254/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-986088" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (284.12s)