=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-499466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-499466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m30.930393445s)
-- stdout --
* [old-k8s-version-499466] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=18966
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18966-3963/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18966-3963/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-499466" primary control-plane node in "old-k8s-version-499466" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0528 21:38:54.607657 64874 out.go:291] Setting OutFile to fd 1 ...
I0528 21:38:54.608080 64874 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 21:38:54.608094 64874 out.go:304] Setting ErrFile to fd 2...
I0528 21:38:54.608099 64874 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 21:38:54.608349 64874 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18966-3963/.minikube/bin
I0528 21:38:54.608928 64874 out.go:298] Setting JSON to false
I0528 21:38:54.610113 64874 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-10","uptime":4878,"bootTime":1716927457,"procs":308,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1060-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0528 21:38:54.610168 64874 start.go:139] virtualization: kvm guest
I0528 21:38:54.612429 64874 out.go:177] * [old-k8s-version-499466] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0528 21:38:54.613610 64874 out.go:177] - MINIKUBE_LOCATION=18966
I0528 21:38:54.613683 64874 notify.go:220] Checking for updates...
I0528 21:38:54.614849 64874 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0528 21:38:54.616250 64874 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18966-3963/kubeconfig
I0528 21:38:54.617495 64874 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18966-3963/.minikube
I0528 21:38:54.618760 64874 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0528 21:38:54.620057 64874 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0528 21:38:54.621692 64874 config.go:182] Loaded profile config "bridge-110727": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 21:38:54.621809 64874 config.go:182] Loaded profile config "cert-expiration-257793": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 21:38:54.621912 64874 config.go:182] Loaded profile config "enable-default-cni-110727": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 21:38:54.622060 64874 driver.go:392] Setting default libvirt URI to qemu:///system
I0528 21:38:54.660357 64874 out.go:177] * Using the kvm2 driver based on user configuration
I0528 21:38:54.661626 64874 start.go:297] selected driver: kvm2
I0528 21:38:54.661639 64874 start.go:901] validating driver "kvm2" against <nil>
I0528 21:38:54.661650 64874 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0528 21:38:54.662496 64874 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 21:38:54.662588 64874 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/18966-3963/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0528 21:38:54.678607 64874 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0528 21:38:54.678658 64874 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0528 21:38:54.678858 64874 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0528 21:38:54.678909 64874 cni.go:84] Creating CNI manager for ""
I0528 21:38:54.678921 64874 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0528 21:38:54.678929 64874 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0528 21:38:54.678977 64874 start.go:340] cluster config:
{Name:old-k8s-version-499466 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-499466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 21:38:54.679068 64874 iso.go:125] acquiring lock: {Name:mk0ef48bd19f4019c3ee87391d15b9afc1d5e8d5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 21:38:54.680725 64874 out.go:177] * Starting "old-k8s-version-499466" primary control-plane node in "old-k8s-version-499466" cluster
I0528 21:38:54.681697 64874 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0528 21:38:54.681731 64874 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0528 21:38:54.681754 64874 cache.go:56] Caching tarball of preloaded images
I0528 21:38:54.681851 64874 preload.go:173] Found /home/jenkins/minikube-integration/18966-3963/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0528 21:38:54.681862 64874 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0528 21:38:54.681963 64874 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/config.json ...
I0528 21:38:54.681987 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/config.json: {Name:mkbf9b6d953414fe7baac9d6851ad2bc3a1f804c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:38:54.682143 64874 start.go:360] acquireMachinesLock for old-k8s-version-499466: {Name:mk6fb3103b370c7f3d9923fd9de7afe2c77239d1 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0528 21:38:54.682177 64874 start.go:364] duration metric: took 16.848µs to acquireMachinesLock for "old-k8s-version-499466"
I0528 21:38:54.682192 64874 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-499466 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18934/minikube-v1.33.1-1716398070-18934-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-499466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0528 21:38:54.682279 64874 start.go:125] createHost starting for "" (driver="kvm2")
I0528 21:38:54.683690 64874 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0528 21:38:54.683834 64874 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0528 21:38:54.683878 64874 main.go:141] libmachine: Launching plugin server for driver kvm2
I0528 21:38:54.698906 64874 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:32977
I0528 21:38:54.699310 64874 main.go:141] libmachine: () Calling .GetVersion
I0528 21:38:54.699819 64874 main.go:141] libmachine: Using API Version 1
I0528 21:38:54.699847 64874 main.go:141] libmachine: () Calling .SetConfigRaw
I0528 21:38:54.700186 64874 main.go:141] libmachine: () Calling .GetMachineName
I0528 21:38:54.700398 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetMachineName
I0528 21:38:54.700584 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:38:54.700744 64874 start.go:159] libmachine.API.Create for "old-k8s-version-499466" (driver="kvm2")
I0528 21:38:54.700780 64874 client.go:168] LocalClient.Create starting
I0528 21:38:54.700817 64874 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem
I0528 21:38:54.700851 64874 main.go:141] libmachine: Decoding PEM data...
I0528 21:38:54.700868 64874 main.go:141] libmachine: Parsing certificate...
I0528 21:38:54.700934 64874 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18966-3963/.minikube/certs/cert.pem
I0528 21:38:54.700962 64874 main.go:141] libmachine: Decoding PEM data...
I0528 21:38:54.700984 64874 main.go:141] libmachine: Parsing certificate...
I0528 21:38:54.701011 64874 main.go:141] libmachine: Running pre-create checks...
I0528 21:38:54.701022 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .PreCreateCheck
I0528 21:38:54.701358 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetConfigRaw
I0528 21:38:54.701720 64874 main.go:141] libmachine: Creating machine...
I0528 21:38:54.701739 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .Create
I0528 21:38:54.701881 64874 main.go:141] libmachine: (old-k8s-version-499466) Creating KVM machine...
I0528 21:38:54.703131 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found existing default KVM network
I0528 21:38:54.704623 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:54.704478 64898 network.go:206] using free private subnet 192.168.39.0/24: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00012df90}
I0528 21:38:54.704651 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | created network xml:
I0528 21:38:54.704664 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <network>
I0528 21:38:54.704670 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <name>mk-old-k8s-version-499466</name>
I0528 21:38:54.704676 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <dns enable='no'/>
I0528 21:38:54.704683 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG |
I0528 21:38:54.704689 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <ip address='192.168.39.1' netmask='255.255.255.0'>
I0528 21:38:54.704697 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <dhcp>
I0528 21:38:54.704703 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | <range start='192.168.39.2' end='192.168.39.253'/>
I0528 21:38:54.704713 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | </dhcp>
I0528 21:38:54.704723 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | </ip>
I0528 21:38:54.704737 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG |
I0528 21:38:54.704750 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | </network>
I0528 21:38:54.704759 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG |
I0528 21:38:54.709953 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | trying to create private KVM network mk-old-k8s-version-499466 192.168.39.0/24...
I0528 21:38:54.781989 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | private KVM network mk-old-k8s-version-499466 192.168.39.0/24 created
I0528 21:38:54.782021 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:54.781949 64898 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/18966-3963/.minikube
I0528 21:38:54.782036 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting up store path in /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466 ...
I0528 21:38:54.782068 64874 main.go:141] libmachine: (old-k8s-version-499466) Building disk image from file:///home/jenkins/minikube-integration/18966-3963/.minikube/cache/iso/amd64/minikube-v1.33.1-1716398070-18934-amd64.iso
I0528 21:38:54.782149 64874 main.go:141] libmachine: (old-k8s-version-499466) Downloading /home/jenkins/minikube-integration/18966-3963/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/18966-3963/.minikube/cache/iso/amd64/minikube-v1.33.1-1716398070-18934-amd64.iso...
I0528 21:38:55.028734 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:55.028629 64898 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa...
I0528 21:38:55.308629 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:55.308484 64898 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/old-k8s-version-499466.rawdisk...
I0528 21:38:55.308669 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Writing magic tar header
I0528 21:38:55.308690 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Writing SSH key tar header
I0528 21:38:55.308703 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:55.308592 64898 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466 ...
I0528 21:38:55.308721 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466
I0528 21:38:55.308738 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466 (perms=drwx------)
I0528 21:38:55.308749 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18966-3963/.minikube/machines
I0528 21:38:55.308762 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18966-3963/.minikube
I0528 21:38:55.308771 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/18966-3963
I0528 21:38:55.308781 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins/minikube-integration/18966-3963/.minikube/machines (perms=drwxr-xr-x)
I0528 21:38:55.308789 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins/minikube-integration/18966-3963/.minikube (perms=drwxr-xr-x)
I0528 21:38:55.308798 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins/minikube-integration/18966-3963 (perms=drwxrwxr-x)
I0528 21:38:55.308806 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0528 21:38:55.308813 64874 main.go:141] libmachine: (old-k8s-version-499466) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0528 21:38:55.308820 64874 main.go:141] libmachine: (old-k8s-version-499466) Creating domain...
I0528 21:38:55.308845 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0528 21:38:55.308855 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home/jenkins
I0528 21:38:55.308860 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Checking permissions on dir: /home
I0528 21:38:55.308866 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Skipping /home - not owner
I0528 21:38:55.310129 64874 main.go:141] libmachine: (old-k8s-version-499466) define libvirt domain using xml:
I0528 21:38:55.310153 64874 main.go:141] libmachine: (old-k8s-version-499466) <domain type='kvm'>
I0528 21:38:55.310165 64874 main.go:141] libmachine: (old-k8s-version-499466) <name>old-k8s-version-499466</name>
I0528 21:38:55.310172 64874 main.go:141] libmachine: (old-k8s-version-499466) <memory unit='MiB'>2200</memory>
I0528 21:38:55.310181 64874 main.go:141] libmachine: (old-k8s-version-499466) <vcpu>2</vcpu>
I0528 21:38:55.310187 64874 main.go:141] libmachine: (old-k8s-version-499466) <features>
I0528 21:38:55.310196 64874 main.go:141] libmachine: (old-k8s-version-499466) <acpi/>
I0528 21:38:55.310204 64874 main.go:141] libmachine: (old-k8s-version-499466) <apic/>
I0528 21:38:55.310214 64874 main.go:141] libmachine: (old-k8s-version-499466) <pae/>
I0528 21:38:55.310235 64874 main.go:141] libmachine: (old-k8s-version-499466)
I0528 21:38:55.310249 64874 main.go:141] libmachine: (old-k8s-version-499466) </features>
I0528 21:38:55.310261 64874 main.go:141] libmachine: (old-k8s-version-499466) <cpu mode='host-passthrough'>
I0528 21:38:55.310270 64874 main.go:141] libmachine: (old-k8s-version-499466)
I0528 21:38:55.310280 64874 main.go:141] libmachine: (old-k8s-version-499466) </cpu>
I0528 21:38:55.310310 64874 main.go:141] libmachine: (old-k8s-version-499466) <os>
I0528 21:38:55.310335 64874 main.go:141] libmachine: (old-k8s-version-499466) <type>hvm</type>
I0528 21:38:55.310359 64874 main.go:141] libmachine: (old-k8s-version-499466) <boot dev='cdrom'/>
I0528 21:38:55.310370 64874 main.go:141] libmachine: (old-k8s-version-499466) <boot dev='hd'/>
I0528 21:38:55.310406 64874 main.go:141] libmachine: (old-k8s-version-499466) <bootmenu enable='no'/>
I0528 21:38:55.310426 64874 main.go:141] libmachine: (old-k8s-version-499466) </os>
I0528 21:38:55.310440 64874 main.go:141] libmachine: (old-k8s-version-499466) <devices>
I0528 21:38:55.310452 64874 main.go:141] libmachine: (old-k8s-version-499466) <disk type='file' device='cdrom'>
I0528 21:38:55.310481 64874 main.go:141] libmachine: (old-k8s-version-499466) <source file='/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/boot2docker.iso'/>
I0528 21:38:55.310510 64874 main.go:141] libmachine: (old-k8s-version-499466) <target dev='hdc' bus='scsi'/>
I0528 21:38:55.310519 64874 main.go:141] libmachine: (old-k8s-version-499466) <readonly/>
I0528 21:38:55.310524 64874 main.go:141] libmachine: (old-k8s-version-499466) </disk>
I0528 21:38:55.310530 64874 main.go:141] libmachine: (old-k8s-version-499466) <disk type='file' device='disk'>
I0528 21:38:55.310539 64874 main.go:141] libmachine: (old-k8s-version-499466) <driver name='qemu' type='raw' cache='default' io='threads' />
I0528 21:38:55.310550 64874 main.go:141] libmachine: (old-k8s-version-499466) <source file='/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/old-k8s-version-499466.rawdisk'/>
I0528 21:38:55.310556 64874 main.go:141] libmachine: (old-k8s-version-499466) <target dev='hda' bus='virtio'/>
I0528 21:38:55.310562 64874 main.go:141] libmachine: (old-k8s-version-499466) </disk>
I0528 21:38:55.310570 64874 main.go:141] libmachine: (old-k8s-version-499466) <interface type='network'>
I0528 21:38:55.310579 64874 main.go:141] libmachine: (old-k8s-version-499466) <source network='mk-old-k8s-version-499466'/>
I0528 21:38:55.310584 64874 main.go:141] libmachine: (old-k8s-version-499466) <model type='virtio'/>
I0528 21:38:55.310592 64874 main.go:141] libmachine: (old-k8s-version-499466) </interface>
I0528 21:38:55.310596 64874 main.go:141] libmachine: (old-k8s-version-499466) <interface type='network'>
I0528 21:38:55.310603 64874 main.go:141] libmachine: (old-k8s-version-499466) <source network='default'/>
I0528 21:38:55.310608 64874 main.go:141] libmachine: (old-k8s-version-499466) <model type='virtio'/>
I0528 21:38:55.310615 64874 main.go:141] libmachine: (old-k8s-version-499466) </interface>
I0528 21:38:55.310625 64874 main.go:141] libmachine: (old-k8s-version-499466) <serial type='pty'>
I0528 21:38:55.310633 64874 main.go:141] libmachine: (old-k8s-version-499466) <target port='0'/>
I0528 21:38:55.310638 64874 main.go:141] libmachine: (old-k8s-version-499466) </serial>
I0528 21:38:55.310644 64874 main.go:141] libmachine: (old-k8s-version-499466) <console type='pty'>
I0528 21:38:55.310649 64874 main.go:141] libmachine: (old-k8s-version-499466) <target type='serial' port='0'/>
I0528 21:38:55.310667 64874 main.go:141] libmachine: (old-k8s-version-499466) </console>
I0528 21:38:55.310682 64874 main.go:141] libmachine: (old-k8s-version-499466) <rng model='virtio'>
I0528 21:38:55.310697 64874 main.go:141] libmachine: (old-k8s-version-499466) <backend model='random'>/dev/random</backend>
I0528 21:38:55.310709 64874 main.go:141] libmachine: (old-k8s-version-499466) </rng>
I0528 21:38:55.310720 64874 main.go:141] libmachine: (old-k8s-version-499466)
I0528 21:38:55.310729 64874 main.go:141] libmachine: (old-k8s-version-499466)
I0528 21:38:55.310738 64874 main.go:141] libmachine: (old-k8s-version-499466) </devices>
I0528 21:38:55.310748 64874 main.go:141] libmachine: (old-k8s-version-499466) </domain>
I0528 21:38:55.310761 64874 main.go:141] libmachine: (old-k8s-version-499466)
I0528 21:38:55.314706 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:a1:39:55 in network default
I0528 21:38:55.315234 64874 main.go:141] libmachine: (old-k8s-version-499466) Ensuring networks are active...
I0528 21:38:55.315248 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:55.315917 64874 main.go:141] libmachine: (old-k8s-version-499466) Ensuring network default is active
I0528 21:38:55.316283 64874 main.go:141] libmachine: (old-k8s-version-499466) Ensuring network mk-old-k8s-version-499466 is active
I0528 21:38:55.316787 64874 main.go:141] libmachine: (old-k8s-version-499466) Getting domain xml...
I0528 21:38:55.317513 64874 main.go:141] libmachine: (old-k8s-version-499466) Creating domain...
I0528 21:38:56.565811 64874 main.go:141] libmachine: (old-k8s-version-499466) Waiting to get IP...
I0528 21:38:56.566671 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:56.567133 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:56.567149 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:56.567118 64898 retry.go:31] will retry after 200.872196ms: waiting for machine to come up
I0528 21:38:56.769653 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:56.770217 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:56.770248 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:56.770184 64898 retry.go:31] will retry after 344.626094ms: waiting for machine to come up
I0528 21:38:57.116716 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:57.117308 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:57.117356 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:57.117237 64898 retry.go:31] will retry after 440.780244ms: waiting for machine to come up
I0528 21:38:57.559860 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:57.560412 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:57.560434 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:57.560369 64898 retry.go:31] will retry after 401.740461ms: waiting for machine to come up
I0528 21:38:57.964113 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:57.964629 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:57.964655 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:57.964587 64898 retry.go:31] will retry after 467.681656ms: waiting for machine to come up
I0528 21:38:58.434274 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:58.434884 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:58.434909 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:58.434837 64898 retry.go:31] will retry after 679.609629ms: waiting for machine to come up
I0528 21:38:59.116286 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:38:59.116824 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:38:59.116861 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:38:59.116797 64898 retry.go:31] will retry after 996.850419ms: waiting for machine to come up
I0528 21:39:00.114791 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:00.115341 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:00.115366 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:00.115286 64898 retry.go:31] will retry after 1.476299895s: waiting for machine to come up
I0528 21:39:01.593802 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:01.594281 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:01.594305 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:01.594231 64898 retry.go:31] will retry after 1.585827174s: waiting for machine to come up
I0528 21:39:03.182064 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:03.182664 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:03.182696 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:03.182610 64898 retry.go:31] will retry after 1.652626175s: waiting for machine to come up
I0528 21:39:04.837168 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:04.837640 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:04.837664 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:04.837583 64898 retry.go:31] will retry after 1.77136399s: waiting for machine to come up
I0528 21:39:06.610999 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:06.611517 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:06.611574 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:06.611502 64898 retry.go:31] will retry after 2.795396118s: waiting for machine to come up
I0528 21:39:09.408350 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:09.408905 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:09.408933 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:09.408860 64898 retry.go:31] will retry after 3.638624784s: waiting for machine to come up
I0528 21:39:13.050868 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:13.051550 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find current IP address of domain old-k8s-version-499466 in network mk-old-k8s-version-499466
I0528 21:39:13.051574 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | I0528 21:39:13.051513 64898 retry.go:31] will retry after 5.055044606s: waiting for machine to come up
I0528 21:39:18.107669 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.108141 64874 main.go:141] libmachine: (old-k8s-version-499466) Found IP for machine: 192.168.39.8
I0528 21:39:18.108166 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has current primary IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.108172 64874 main.go:141] libmachine: (old-k8s-version-499466) Reserving static IP address...
I0528 21:39:18.108527 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-499466", mac: "52:54:00:04:bf:9b", ip: "192.168.39.8"} in network mk-old-k8s-version-499466
I0528 21:39:18.183411 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Getting to WaitForSSH function...
I0528 21:39:18.183443 64874 main.go:141] libmachine: (old-k8s-version-499466) Reserved static IP address: 192.168.39.8
I0528 21:39:18.183458 64874 main.go:141] libmachine: (old-k8s-version-499466) Waiting for SSH to be available...
I0528 21:39:18.186165 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.186559 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:minikube Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.186603 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.186747 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Using SSH client type: external
I0528 21:39:18.186775 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Using SSH private key: /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa (-rw-------)
I0528 21:39:18.186820 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.8 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa -p 22] /usr/bin/ssh <nil>}
I0528 21:39:18.186838 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | About to run SSH command:
I0528 21:39:18.186874 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | exit 0
I0528 21:39:18.306039 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | SSH cmd err, output: <nil>:
I0528 21:39:18.306320 64874 main.go:141] libmachine: (old-k8s-version-499466) KVM machine creation complete!
I0528 21:39:18.306609 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetConfigRaw
I0528 21:39:18.307120 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:18.307300 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:18.307474 64874 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0528 21:39:18.307487 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetState
I0528 21:39:18.308849 64874 main.go:141] libmachine: Detecting operating system of created instance...
I0528 21:39:18.308861 64874 main.go:141] libmachine: Waiting for SSH to be available...
I0528 21:39:18.308866 64874 main.go:141] libmachine: Getting to WaitForSSH function...
I0528 21:39:18.308871 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.311175 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.311530 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.311572 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.311670 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.311809 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.311957 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.312117 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.312274 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:18.312455 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:18.312466 64874 main.go:141] libmachine: About to run SSH command:
exit 0
I0528 21:39:18.409413 64874 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0528 21:39:18.409447 64874 main.go:141] libmachine: Detecting the provisioner...
I0528 21:39:18.409458 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.412206 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.412547 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.412575 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.412754 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.412944 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.413102 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.413242 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.413395 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:18.413605 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:18.413616 64874 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0528 21:39:18.514472 64874 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0528 21:39:18.514545 64874 main.go:141] libmachine: found compatible host: buildroot
I0528 21:39:18.514558 64874 main.go:141] libmachine: Provisioning with buildroot...
I0528 21:39:18.514571 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetMachineName
I0528 21:39:18.514806 64874 buildroot.go:166] provisioning hostname "old-k8s-version-499466"
I0528 21:39:18.514829 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetMachineName
I0528 21:39:18.515042 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.517636 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.517955 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.517982 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.518126 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.518307 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.518473 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.518629 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.518789 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:18.518943 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:18.518954 64874 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-499466 && echo "old-k8s-version-499466" | sudo tee /etc/hostname
I0528 21:39:18.634160 64874 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-499466
I0528 21:39:18.634184 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.636822 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.637163 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.637191 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.637408 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.637568 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.637726 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.637881 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.638069 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:18.638279 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:18.638307 64874 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-499466' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-499466/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-499466' | sudo tee -a /etc/hosts;
fi
fi
I0528 21:39:18.743062 64874 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0528 21:39:18.743092 64874 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/18966-3963/.minikube CaCertPath:/home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18966-3963/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18966-3963/.minikube}
I0528 21:39:18.743133 64874 buildroot.go:174] setting up certificates
I0528 21:39:18.743147 64874 provision.go:84] configureAuth start
I0528 21:39:18.743168 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetMachineName
I0528 21:39:18.743450 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetIP
I0528 21:39:18.746359 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.746769 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.746795 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.746959 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.749239 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.749583 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.749611 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.749790 64874 provision.go:143] copyHostCerts
I0528 21:39:18.749869 64874 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-3963/.minikube/ca.pem, removing ...
I0528 21:39:18.749879 64874 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-3963/.minikube/ca.pem
I0528 21:39:18.749940 64874 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18966-3963/.minikube/ca.pem (1078 bytes)
I0528 21:39:18.750019 64874 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-3963/.minikube/cert.pem, removing ...
I0528 21:39:18.750027 64874 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-3963/.minikube/cert.pem
I0528 21:39:18.750051 64874 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18966-3963/.minikube/cert.pem (1123 bytes)
I0528 21:39:18.750098 64874 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-3963/.minikube/key.pem, removing ...
I0528 21:39:18.750104 64874 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-3963/.minikube/key.pem
I0528 21:39:18.750124 64874 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18966-3963/.minikube/key.pem (1675 bytes)
I0528 21:39:18.750177 64874 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18966-3963/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-499466 san=[127.0.0.1 192.168.39.8 localhost minikube old-k8s-version-499466]
I0528 21:39:18.829587 64874 provision.go:177] copyRemoteCerts
I0528 21:39:18.829639 64874 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0528 21:39:18.829662 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.832310 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.832666 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.832696 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.832890 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.833069 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.833242 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.833379 64874 sshutil.go:53] new ssh client: &{IP:192.168.39.8 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa Username:docker}
I0528 21:39:18.911962 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0528 21:39:18.935280 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0528 21:39:18.958743 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0528 21:39:18.983472 64874 provision.go:87] duration metric: took 240.306898ms to configureAuth
I0528 21:39:18.983498 64874 buildroot.go:189] setting minikube options for container-runtime
I0528 21:39:18.983702 64874 config.go:182] Loaded profile config "old-k8s-version-499466": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0528 21:39:18.983775 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:18.986539 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.986853 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:18.986881 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:18.987048 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:18.987235 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.987420 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:18.987587 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:18.987789 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:18.987989 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:18.988008 64874 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0528 21:39:19.245474 64874 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0528 21:39:19.245496 64874 main.go:141] libmachine: Checking connection to Docker...
I0528 21:39:19.245508 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetURL
I0528 21:39:19.246753 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | Using libvirt version 6000000
I0528 21:39:19.249286 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.249683 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.249713 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.249846 64874 main.go:141] libmachine: Docker is up and running!
I0528 21:39:19.249866 64874 main.go:141] libmachine: Reticulating splines...
I0528 21:39:19.249871 64874 client.go:171] duration metric: took 24.549081697s to LocalClient.Create
I0528 21:39:19.249891 64874 start.go:167] duration metric: took 24.549148529s to libmachine.API.Create "old-k8s-version-499466"
I0528 21:39:19.249901 64874 start.go:293] postStartSetup for "old-k8s-version-499466" (driver="kvm2")
I0528 21:39:19.249913 64874 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0528 21:39:19.249934 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:19.250235 64874 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0528 21:39:19.250263 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:19.252519 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.252870 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.252899 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.253018 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:19.253207 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:19.253377 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:19.253495 64874 sshutil.go:53] new ssh client: &{IP:192.168.39.8 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa Username:docker}
I0528 21:39:19.333872 64874 ssh_runner.go:195] Run: cat /etc/os-release
I0528 21:39:19.338215 64874 info.go:137] Remote host: Buildroot 2023.02.9
I0528 21:39:19.338241 64874 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-3963/.minikube/addons for local assets ...
I0528 21:39:19.338308 64874 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-3963/.minikube/files for local assets ...
I0528 21:39:19.338404 64874 filesync.go:149] local asset: /home/jenkins/minikube-integration/18966-3963/.minikube/files/etc/ssl/certs/117602.pem -> 117602.pem in /etc/ssl/certs
I0528 21:39:19.338494 64874 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0528 21:39:19.347749 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/files/etc/ssl/certs/117602.pem --> /etc/ssl/certs/117602.pem (1708 bytes)
I0528 21:39:19.372165 64874 start.go:296] duration metric: took 122.252031ms for postStartSetup
I0528 21:39:19.372205 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetConfigRaw
I0528 21:39:19.372755 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetIP
I0528 21:39:19.375732 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.376118 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.376146 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.376409 64874 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/config.json ...
I0528 21:39:19.376568 64874 start.go:128] duration metric: took 24.694280834s to createHost
I0528 21:39:19.376587 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:19.379011 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.379352 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.379386 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.379519 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:19.379674 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:19.379791 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:19.379923 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:19.380056 64874 main.go:141] libmachine: Using SSH client type: native
I0528 21:39:19.380216 64874 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82d800] 0x830560 <nil> [] 0s} 192.168.39.8 22 <nil> <nil>}
I0528 21:39:19.380232 64874 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0528 21:39:19.478360 64874 main.go:141] libmachine: SSH cmd err, output: <nil>: 1716932359.452828168
I0528 21:39:19.478379 64874 fix.go:216] guest clock: 1716932359.452828168
I0528 21:39:19.478385 64874 fix.go:229] Guest: 2024-05-28 21:39:19.452828168 +0000 UTC Remote: 2024-05-28 21:39:19.376577347 +0000 UTC m=+24.802909608 (delta=76.250821ms)
I0528 21:39:19.478461 64874 fix.go:200] guest clock delta is within tolerance: 76.250821ms
I0528 21:39:19.478467 64874 start.go:83] releasing machines lock for "old-k8s-version-499466", held for 24.796283279s
I0528 21:39:19.478486 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:19.478759 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetIP
I0528 21:39:19.481177 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.481565 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.481595 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.481712 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:19.482204 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:19.482371 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .DriverName
I0528 21:39:19.482470 64874 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0528 21:39:19.482511 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:19.482567 64874 ssh_runner.go:195] Run: cat /version.json
I0528 21:39:19.482588 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHHostname
I0528 21:39:19.485016 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.485311 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.485449 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.485524 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.485585 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:19.485737 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:19.485755 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:19.485756 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:19.485945 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHPort
I0528 21:39:19.485954 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:19.486091 64874 sshutil.go:53] new ssh client: &{IP:192.168.39.8 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa Username:docker}
I0528 21:39:19.486161 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHKeyPath
I0528 21:39:19.486283 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetSSHUsername
I0528 21:39:19.486446 64874 sshutil.go:53] new ssh client: &{IP:192.168.39.8 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/18966-3963/.minikube/machines/old-k8s-version-499466/id_rsa Username:docker}
I0528 21:39:19.572287 64874 ssh_runner.go:195] Run: systemctl --version
I0528 21:39:19.594199 64874 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0528 21:39:19.755553 64874 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0528 21:39:19.762668 64874 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0528 21:39:19.762754 64874 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0528 21:39:19.779606 64874 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0528 21:39:19.779627 64874 start.go:494] detecting cgroup driver to use...
I0528 21:39:19.779679 64874 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0528 21:39:19.799036 64874 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0528 21:39:19.815568 64874 docker.go:217] disabling cri-docker service (if available) ...
I0528 21:39:19.815638 64874 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0528 21:39:19.831729 64874 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0528 21:39:19.845508 64874 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0528 21:39:19.969376 64874 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0528 21:39:20.111944 64874 docker.go:233] disabling docker service ...
I0528 21:39:20.112026 64874 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0528 21:39:20.127282 64874 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0528 21:39:20.144615 64874 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0528 21:39:20.286243 64874 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0528 21:39:20.430024 64874 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0528 21:39:20.446574 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0528 21:39:20.466965 64874 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0528 21:39:20.467038 64874 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 21:39:20.481150 64874 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0528 21:39:20.481204 64874 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 21:39:20.492078 64874 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0528 21:39:20.502178 64874 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0528 21:39:20.512853 64874 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0528 21:39:20.523587 64874 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0528 21:39:20.533890 64874 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0528 21:39:20.533944 64874 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0528 21:39:20.549526 64874 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0528 21:39:20.561211 64874 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 21:39:20.693136 64874 ssh_runner.go:195] Run: sudo systemctl restart crio
I0528 21:39:20.835430 64874 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0528 21:39:20.835538 64874 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0528 21:39:20.842129 64874 start.go:562] Will wait 60s for crictl version
I0528 21:39:20.842180 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:20.846354 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0528 21:39:20.889604 64874 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0528 21:39:20.889663 64874 ssh_runner.go:195] Run: crio --version
I0528 21:39:20.921984 64874 ssh_runner.go:195] Run: crio --version
I0528 21:39:20.954289 64874 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0528 21:39:20.955485 64874 main.go:141] libmachine: (old-k8s-version-499466) Calling .GetIP
I0528 21:39:20.958338 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:20.958654 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:04:bf:9b", ip: ""} in network mk-old-k8s-version-499466: {Iface:virbr1 ExpiryTime:2024-05-28 22:39:09 +0000 UTC Type:0 Mac:52:54:00:04:bf:9b Iaid: IPaddr:192.168.39.8 Prefix:24 Hostname:old-k8s-version-499466 Clientid:01:52:54:00:04:bf:9b}
I0528 21:39:20.958684 64874 main.go:141] libmachine: (old-k8s-version-499466) DBG | domain old-k8s-version-499466 has defined IP address 192.168.39.8 and MAC address 52:54:00:04:bf:9b in network mk-old-k8s-version-499466
I0528 21:39:20.958829 64874 ssh_runner.go:195] Run: grep 192.168.39.1 host.minikube.internal$ /etc/hosts
I0528 21:39:20.963646 64874 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.39.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 21:39:20.977850 64874 kubeadm.go:877] updating cluster {Name:old-k8s-version-499466 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18934/minikube-v1.33.1-1716398070-18934-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-499466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.8 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mou
ntPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0528 21:39:20.977980 64874 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0528 21:39:20.978035 64874 ssh_runner.go:195] Run: sudo crictl images --output json
I0528 21:39:21.017449 64874 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0528 21:39:21.017501 64874 ssh_runner.go:195] Run: which lz4
I0528 21:39:21.021886 64874 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0528 21:39:21.026231 64874 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0528 21:39:21.026261 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0528 21:39:22.677352 64874 crio.go:462] duration metric: took 1.655490738s to copy over tarball
I0528 21:39:22.677416 64874 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0528 21:39:25.553028 64874 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.875582718s)
I0528 21:39:25.553053 64874 crio.go:469] duration metric: took 2.875674993s to extract the tarball
I0528 21:39:25.553061 64874 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0528 21:39:25.597229 64874 ssh_runner.go:195] Run: sudo crictl images --output json
I0528 21:39:25.650549 64874 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0528 21:39:25.650576 64874 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0528 21:39:25.650677 64874 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0528 21:39:25.650706 64874 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0528 21:39:25.650709 64874 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0528 21:39:25.650678 64874 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0528 21:39:25.650678 64874 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0528 21:39:25.650695 64874 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0528 21:39:25.650741 64874 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0528 21:39:25.650738 64874 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0528 21:39:25.652470 64874 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0528 21:39:25.653355 64874 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0528 21:39:25.653387 64874 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0528 21:39:25.653425 64874 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0528 21:39:25.653454 64874 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0528 21:39:25.653677 64874 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0528 21:39:25.653621 64874 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0528 21:39:25.653943 64874 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0528 21:39:25.783658 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0528 21:39:25.794568 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0528 21:39:25.816058 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0528 21:39:25.817184 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0528 21:39:25.828193 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0528 21:39:25.833281 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0528 21:39:25.898222 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0528 21:39:25.902158 64874 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0528 21:39:25.902201 64874 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0528 21:39:25.902241 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016325 64874 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0528 21:39:26.016369 64874 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0528 21:39:26.016400 64874 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0528 21:39:26.016414 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016419 64874 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0528 21:39:26.016421 64874 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0528 21:39:26.016437 64874 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0528 21:39:26.016469 64874 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0528 21:39:26.016472 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0528 21:39:26.016445 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016473 64874 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0528 21:39:26.016398 64874 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0528 21:39:26.016515 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016524 64874 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0528 21:39:26.016444 64874 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0528 21:39:26.016549 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016561 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.016493 64874 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0528 21:39:26.016587 64874 ssh_runner.go:195] Run: which crictl
I0528 21:39:26.034881 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0528 21:39:26.034961 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0528 21:39:26.034964 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0528 21:39:26.035006 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0528 21:39:26.112469 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0528 21:39:26.112595 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0528 21:39:26.112641 64874 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0528 21:39:26.184887 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0528 21:39:26.184960 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0528 21:39:26.185024 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0528 21:39:26.185214 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0528 21:39:26.202751 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0528 21:39:26.202815 64874 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0528 21:39:26.618275 64874 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0528 21:39:26.761312 64874 cache_images.go:92] duration metric: took 1.110713335s to LoadCachedImages
W0528 21:39:26.761397 64874 out.go:239] X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0: no such file or directory
X Unable to load cached images: loading cached images: stat /home/jenkins/minikube-integration/18966-3963/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0: no such file or directory
I0528 21:39:26.761417 64874 kubeadm.go:928] updating node { 192.168.39.8 8443 v1.20.0 crio true true} ...
I0528 21:39:26.761548 64874 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-499466 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.39.8
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-499466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0528 21:39:26.761626 64874 ssh_runner.go:195] Run: crio config
I0528 21:39:26.819662 64874 cni.go:84] Creating CNI manager for ""
I0528 21:39:26.819686 64874 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0528 21:39:26.819702 64874 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0528 21:39:26.819728 64874 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.8 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-499466 NodeName:old-k8s-version-499466 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.8"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.39.8 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0528 21:39:26.819913 64874 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.39.8
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-499466"
kubeletExtraArgs:
node-ip: 192.168.39.8
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.39.8"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0528 21:39:26.819976 64874 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0528 21:39:26.839577 64874 binaries.go:44] Found k8s binaries, skipping transfer
I0528 21:39:26.839644 64874 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0528 21:39:26.849714 64874 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (428 bytes)
I0528 21:39:26.867170 64874 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0528 21:39:26.886183 64874 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2117 bytes)
I0528 21:39:26.904995 64874 ssh_runner.go:195] Run: grep 192.168.39.8 control-plane.minikube.internal$ /etc/hosts
I0528 21:39:26.909387 64874 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.39.8 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 21:39:26.921662 64874 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 21:39:27.062057 64874 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0528 21:39:27.080446 64874 certs.go:68] Setting up /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466 for IP: 192.168.39.8
I0528 21:39:27.080463 64874 certs.go:194] generating shared ca certs ...
I0528 21:39:27.080475 64874 certs.go:226] acquiring lock for ca certs: {Name:mkf081a2e878fd451cfbdf01dc28a5f7a6a3abc9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.080590 64874 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18966-3963/.minikube/ca.key
I0528 21:39:27.080622 64874 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18966-3963/.minikube/proxy-client-ca.key
I0528 21:39:27.080632 64874 certs.go:256] generating profile certs ...
I0528 21:39:27.080684 64874 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.key
I0528 21:39:27.080697 64874 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.crt with IP's: []
I0528 21:39:27.378529 64874 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.crt ...
I0528 21:39:27.378566 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.crt: {Name:mk9ce377fa8e4769738cf95d9a55efd02085902b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.378751 64874 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.key ...
I0528 21:39:27.378772 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/client.key: {Name:mk4a94797a4a087660b6c9541333ad93a97da473 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.378902 64874 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key.2337190f
I0528 21:39:27.378923 64874 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt.2337190f with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.39.8]
I0528 21:39:27.447720 64874 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt.2337190f ...
I0528 21:39:27.447746 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt.2337190f: {Name:mk98ac8132284941449a339fe624a11abdb8bbc9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.476157 64874 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key.2337190f ...
I0528 21:39:27.476192 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key.2337190f: {Name:mkc4fa3ada3de02e43b108ae3273ec46b870742b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.476335 64874 certs.go:381] copying /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt.2337190f -> /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt
I0528 21:39:27.476440 64874 certs.go:385] copying /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key.2337190f -> /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key
I0528 21:39:27.476602 64874 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.key
I0528 21:39:27.476624 64874 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.crt with IP's: []
I0528 21:39:27.997407 64874 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.crt ...
I0528 21:39:27.997437 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.crt: {Name:mk7da2d3da24412f2ae7539ac3c6a2f9240f9317 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.997581 64874 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.key ...
I0528 21:39:27.997593 64874 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.key: {Name:mk1e14433d517b653aa5d93dc2bbfea00347a1d4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 21:39:27.997778 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/11760.pem (1338 bytes)
W0528 21:39:27.997825 64874 certs.go:480] ignoring /home/jenkins/minikube-integration/18966-3963/.minikube/certs/11760_empty.pem, impossibly tiny 0 bytes
I0528 21:39:27.997835 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca-key.pem (1675 bytes)
I0528 21:39:27.997856 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/ca.pem (1078 bytes)
I0528 21:39:27.997880 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/cert.pem (1123 bytes)
I0528 21:39:27.997900 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/certs/key.pem (1675 bytes)
I0528 21:39:27.997948 64874 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-3963/.minikube/files/etc/ssl/certs/117602.pem (1708 bytes)
I0528 21:39:27.998577 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0528 21:39:28.030220 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0528 21:39:28.055705 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0528 21:39:28.096165 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0528 21:39:28.134093 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0528 21:39:28.161112 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0528 21:39:28.203156 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0528 21:39:28.228670 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/old-k8s-version-499466/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0528 21:39:28.276705 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0528 21:39:28.305273 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/certs/11760.pem --> /usr/share/ca-certificates/11760.pem (1338 bytes)
I0528 21:39:28.334957 64874 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-3963/.minikube/files/etc/ssl/certs/117602.pem --> /usr/share/ca-certificates/117602.pem (1708 bytes)
I0528 21:39:28.360036 64874 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0528 21:39:28.377903 64874 ssh_runner.go:195] Run: openssl version
I0528 21:39:28.384887 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/11760.pem && ln -fs /usr/share/ca-certificates/11760.pem /etc/ssl/certs/11760.pem"
I0528 21:39:28.398944 64874 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/11760.pem
I0528 21:39:28.403508 64874 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 May 28 20:34 /usr/share/ca-certificates/11760.pem
I0528 21:39:28.403557 64874 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/11760.pem
I0528 21:39:28.410090 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/11760.pem /etc/ssl/certs/51391683.0"
I0528 21:39:28.420781 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/117602.pem && ln -fs /usr/share/ca-certificates/117602.pem /etc/ssl/certs/117602.pem"
I0528 21:39:28.431440 64874 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/117602.pem
I0528 21:39:28.435650 64874 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 May 28 20:34 /usr/share/ca-certificates/117602.pem
I0528 21:39:28.435690 64874 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/117602.pem
I0528 21:39:28.441325 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/117602.pem /etc/ssl/certs/3ec20f2e.0"
I0528 21:39:28.451798 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0528 21:39:28.462697 64874 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0528 21:39:28.467808 64874 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 May 28 20:22 /usr/share/ca-certificates/minikubeCA.pem
I0528 21:39:28.467847 64874 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0528 21:39:28.473547 64874 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0528 21:39:28.484265 64874 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0528 21:39:28.488324 64874 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0528 21:39:28.488380 64874 kubeadm.go:391] StartCluster: {Name:old-k8s-version-499466 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18934/minikube-v1.33.1-1716398070-18934-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-499466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.8 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountP
ort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 21:39:28.488485 64874 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0528 21:39:28.488536 64874 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0528 21:39:28.528672 64874 cri.go:89] found id: ""
I0528 21:39:28.528753 64874 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0528 21:39:28.539887 64874 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0528 21:39:28.550542 64874 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0528 21:39:28.561103 64874 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0528 21:39:28.561124 64874 kubeadm.go:156] found existing configuration files:
I0528 21:39:28.561170 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0528 21:39:28.571092 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0528 21:39:28.571142 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0528 21:39:28.580641 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0528 21:39:28.590332 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0528 21:39:28.590390 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0528 21:39:28.600277 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0528 21:39:28.609686 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0528 21:39:28.609728 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0528 21:39:28.618994 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0528 21:39:28.630696 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0528 21:39:28.630736 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0528 21:39:28.640055 64874 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0528 21:39:29.046169 64874 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0528 21:41:27.357907 64874 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0528 21:41:27.358039 64874 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0528 21:41:27.359384 64874 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0528 21:41:27.359445 64874 kubeadm.go:309] [preflight] Running pre-flight checks
I0528 21:41:27.359557 64874 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0528 21:41:27.359705 64874 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0528 21:41:27.359859 64874 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0528 21:41:27.359954 64874 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0528 21:41:27.361833 64874 out.go:204] - Generating certificates and keys ...
I0528 21:41:27.361924 64874 kubeadm.go:309] [certs] Using existing ca certificate authority
I0528 21:41:27.361984 64874 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0528 21:41:27.362071 64874 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0528 21:41:27.362151 64874 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0528 21:41:27.362225 64874 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0528 21:41:27.362296 64874 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0528 21:41:27.362357 64874 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0528 21:41:27.362492 64874 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
I0528 21:41:27.362583 64874 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0528 21:41:27.362714 64874 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
I0528 21:41:27.362799 64874 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0528 21:41:27.362884 64874 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0528 21:41:27.362949 64874 kubeadm.go:309] [certs] Generating "sa" key and public key
I0528 21:41:27.363043 64874 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0528 21:41:27.363131 64874 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0528 21:41:27.363221 64874 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0528 21:41:27.363325 64874 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0528 21:41:27.363387 64874 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0528 21:41:27.363516 64874 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0528 21:41:27.363594 64874 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0528 21:41:27.363629 64874 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0528 21:41:27.363711 64874 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0528 21:41:27.365091 64874 out.go:204] - Booting up control plane ...
I0528 21:41:27.365206 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0528 21:41:27.365303 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0528 21:41:27.365370 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0528 21:41:27.365452 64874 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0528 21:41:27.365594 64874 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0528 21:41:27.365658 64874 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0528 21:41:27.365766 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:41:27.366017 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:41:27.366098 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:41:27.366347 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:41:27.366409 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:41:27.366569 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:41:27.366631 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:41:27.366789 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:41:27.366848 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:41:27.367068 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:41:27.367079 64874 kubeadm.go:309]
I0528 21:41:27.367113 64874 kubeadm.go:309] Unfortunately, an error has occurred:
I0528 21:41:27.367149 64874 kubeadm.go:309] timed out waiting for the condition
I0528 21:41:27.367157 64874 kubeadm.go:309]
I0528 21:41:27.367186 64874 kubeadm.go:309] This error is likely caused by:
I0528 21:41:27.367220 64874 kubeadm.go:309] - The kubelet is not running
I0528 21:41:27.367315 64874 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0528 21:41:27.367322 64874 kubeadm.go:309]
I0528 21:41:27.367440 64874 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0528 21:41:27.367479 64874 kubeadm.go:309] - 'systemctl status kubelet'
I0528 21:41:27.367524 64874 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0528 21:41:27.367531 64874 kubeadm.go:309]
I0528 21:41:27.367644 64874 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0528 21:41:27.367738 64874 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0528 21:41:27.367747 64874 kubeadm.go:309]
I0528 21:41:27.367837 64874 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0528 21:41:27.367963 64874 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0528 21:41:27.368075 64874 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0528 21:41:27.368176 64874 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0528 21:41:27.368202 64874 kubeadm.go:309]
W0528 21:41:27.368304 64874 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-499466] and IPs [192.168.39.8 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0528 21:41:27.368348 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0528 21:41:27.872271 64874 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0528 21:41:27.887418 64874 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0528 21:41:27.897087 64874 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0528 21:41:27.897105 64874 kubeadm.go:156] found existing configuration files:
I0528 21:41:27.897141 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0528 21:41:27.906553 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0528 21:41:27.906593 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0528 21:41:27.916150 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0528 21:41:27.925396 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0528 21:41:27.925441 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0528 21:41:27.934862 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0528 21:41:27.943971 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0528 21:41:27.944004 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0528 21:41:27.953334 64874 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0528 21:41:27.961801 64874 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0528 21:41:27.961845 64874 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0528 21:41:27.970614 64874 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0528 21:41:28.038778 64874 kubeadm.go:309] [init] Using Kubernetes version: v1.20.0
I0528 21:41:28.038867 64874 kubeadm.go:309] [preflight] Running pre-flight checks
I0528 21:41:28.181089 64874 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0528 21:41:28.181252 64874 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0528 21:41:28.181397 64874 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0528 21:41:28.353859 64874 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0528 21:41:28.355936 64874 out.go:204] - Generating certificates and keys ...
I0528 21:41:28.356051 64874 kubeadm.go:309] [certs] Using existing ca certificate authority
I0528 21:41:28.356147 64874 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0528 21:41:28.356251 64874 kubeadm.go:309] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0528 21:41:28.356351 64874 kubeadm.go:309] [certs] Using existing front-proxy-ca certificate authority
I0528 21:41:28.356468 64874 kubeadm.go:309] [certs] Using existing front-proxy-client certificate and key on disk
I0528 21:41:28.356556 64874 kubeadm.go:309] [certs] Using existing etcd/ca certificate authority
I0528 21:41:28.356656 64874 kubeadm.go:309] [certs] Using existing etcd/server certificate and key on disk
I0528 21:41:28.356885 64874 kubeadm.go:309] [certs] Using existing etcd/peer certificate and key on disk
I0528 21:41:28.357243 64874 kubeadm.go:309] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0528 21:41:28.357574 64874 kubeadm.go:309] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0528 21:41:28.357643 64874 kubeadm.go:309] [certs] Using the existing "sa" key
I0528 21:41:28.357735 64874 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0528 21:41:28.554078 64874 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0528 21:41:28.966148 64874 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0528 21:41:29.468187 64874 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0528 21:41:29.622446 64874 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0528 21:41:29.636848 64874 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0528 21:41:29.637898 64874 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0528 21:41:29.637971 64874 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0528 21:41:29.806864 64874 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0528 21:41:29.808447 64874 out.go:204] - Booting up control plane ...
I0528 21:41:29.808577 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0528 21:41:29.812249 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0528 21:41:29.813780 64874 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0528 21:41:29.814431 64874 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0528 21:41:29.816692 64874 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0528 21:42:09.819448 64874 kubeadm.go:309] [kubelet-check] Initial timeout of 40s passed.
I0528 21:42:09.819869 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:42:09.820057 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:42:14.820598 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:42:14.820855 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:42:24.821302 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:42:24.821481 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:42:44.820414 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:42:44.820609 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:43:24.820265 64874 kubeadm.go:309] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0528 21:43:24.820541 64874 kubeadm.go:309] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0528 21:43:24.820565 64874 kubeadm.go:309]
I0528 21:43:24.820621 64874 kubeadm.go:309] Unfortunately, an error has occurred:
I0528 21:43:24.820679 64874 kubeadm.go:309] timed out waiting for the condition
I0528 21:43:24.820689 64874 kubeadm.go:309]
I0528 21:43:24.820742 64874 kubeadm.go:309] This error is likely caused by:
I0528 21:43:24.820807 64874 kubeadm.go:309] - The kubelet is not running
I0528 21:43:24.820965 64874 kubeadm.go:309] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0528 21:43:24.820984 64874 kubeadm.go:309]
I0528 21:43:24.821082 64874 kubeadm.go:309] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0528 21:43:24.821115 64874 kubeadm.go:309] - 'systemctl status kubelet'
I0528 21:43:24.821142 64874 kubeadm.go:309] - 'journalctl -xeu kubelet'
I0528 21:43:24.821146 64874 kubeadm.go:309]
I0528 21:43:24.821250 64874 kubeadm.go:309] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0528 21:43:24.821338 64874 kubeadm.go:309] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0528 21:43:24.821360 64874 kubeadm.go:309]
I0528 21:43:24.821464 64874 kubeadm.go:309] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0528 21:43:24.821591 64874 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0528 21:43:24.821691 64874 kubeadm.go:309] Once you have found the failing container, you can inspect its logs with:
I0528 21:43:24.821802 64874 kubeadm.go:309] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0528 21:43:24.821814 64874 kubeadm.go:309]
I0528 21:43:24.822736 64874 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0528 21:43:24.822854 64874 kubeadm.go:309] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0528 21:43:24.822947 64874 kubeadm.go:309] To see the stack trace of this error execute with --v=5 or higher
I0528 21:43:24.823004 64874 kubeadm.go:393] duration metric: took 3m56.334627812s to StartCluster
I0528 21:43:24.823060 64874 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 21:43:24.823112 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 21:43:24.872331 64874 cri.go:89] found id: ""
I0528 21:43:24.872359 64874 logs.go:276] 0 containers: []
W0528 21:43:24.872366 64874 logs.go:278] No container was found matching "kube-apiserver"
I0528 21:43:24.872372 64874 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 21:43:24.872424 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 21:43:24.917358 64874 cri.go:89] found id: ""
I0528 21:43:24.917387 64874 logs.go:276] 0 containers: []
W0528 21:43:24.917395 64874 logs.go:278] No container was found matching "etcd"
I0528 21:43:24.917401 64874 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 21:43:24.917446 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 21:43:24.959747 64874 cri.go:89] found id: ""
I0528 21:43:24.959781 64874 logs.go:276] 0 containers: []
W0528 21:43:24.959791 64874 logs.go:278] No container was found matching "coredns"
I0528 21:43:24.959798 64874 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 21:43:24.959847 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 21:43:25.000522 64874 cri.go:89] found id: ""
I0528 21:43:25.000548 64874 logs.go:276] 0 containers: []
W0528 21:43:25.000559 64874 logs.go:278] No container was found matching "kube-scheduler"
I0528 21:43:25.000566 64874 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 21:43:25.000626 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 21:43:25.042166 64874 cri.go:89] found id: ""
I0528 21:43:25.042191 64874 logs.go:276] 0 containers: []
W0528 21:43:25.042200 64874 logs.go:278] No container was found matching "kube-proxy"
I0528 21:43:25.042208 64874 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 21:43:25.042269 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 21:43:25.076228 64874 cri.go:89] found id: ""
I0528 21:43:25.076258 64874 logs.go:276] 0 containers: []
W0528 21:43:25.076268 64874 logs.go:278] No container was found matching "kube-controller-manager"
I0528 21:43:25.076275 64874 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 21:43:25.076332 64874 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 21:43:25.117156 64874 cri.go:89] found id: ""
I0528 21:43:25.117189 64874 logs.go:276] 0 containers: []
W0528 21:43:25.117199 64874 logs.go:278] No container was found matching "kindnet"
I0528 21:43:25.117210 64874 logs.go:123] Gathering logs for kubelet ...
I0528 21:43:25.117224 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0528 21:43:25.165693 64874 logs.go:123] Gathering logs for dmesg ...
I0528 21:43:25.165722 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 21:43:25.179297 64874 logs.go:123] Gathering logs for describe nodes ...
I0528 21:43:25.179325 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0528 21:43:25.351513 64874 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0528 21:43:25.351535 64874 logs.go:123] Gathering logs for CRI-O ...
I0528 21:43:25.351550 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 21:43:25.450877 64874 logs.go:123] Gathering logs for container status ...
I0528 21:43:25.450910 64874 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0528 21:43:25.489435 64874 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0528 21:43:25.489477 64874 out.go:239] *
*
W0528 21:43:25.489540 64874 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0528 21:43:25.489570 64874 out.go:239] *
*
W0528 21:43:25.490447 64874 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0528 21:43:25.493547 64874 out.go:177]
W0528 21:43:25.494545 64874 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0528 21:43:25.494597 64874 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0528 21:43:25.494620 64874 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0528 21:43:25.495814 64874 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-499466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-499466 -n old-k8s-version-499466
E0528 21:43:25.573266 11760 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18966-3963/.minikube/profiles/flannel-110727/client.crt: no such file or directory
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-499466 -n old-k8s-version-499466: exit status 6 (213.938163ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0528 21:43:25.750621 69461 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-499466" does not appear in /home/jenkins/minikube-integration/18966-3963/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-499466" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (271.20s)