=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-278865 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
E0815 18:26:15.296171 20219 cert_rotation.go:171] "Unhandled Error" err="key failed with : open /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/addons-973562/client.crt: no such file or directory" logger="UnhandledError"
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-278865 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m56.983648251s)
-- stdout --
* [old-k8s-version-278865] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19450
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19450-13013/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19450-13013/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-278865" primary control-plane node in "old-k8s-version-278865" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0815 18:25:53.788893 64368 out.go:345] Setting OutFile to fd 1 ...
I0815 18:25:53.788979 64368 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0815 18:25:53.788984 64368 out.go:358] Setting ErrFile to fd 2...
I0815 18:25:53.788989 64368 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0815 18:25:53.789159 64368 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19450-13013/.minikube/bin
I0815 18:25:53.789706 64368 out.go:352] Setting JSON to false
I0815 18:25:53.790584 64368 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-10","uptime":7700,"bootTime":1723738654,"procs":211,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1066-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0815 18:25:53.790636 64368 start.go:139] virtualization: kvm guest
I0815 18:25:53.792750 64368 out.go:177] * [old-k8s-version-278865] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0815 18:25:53.794152 64368 out.go:177] - MINIKUBE_LOCATION=19450
I0815 18:25:53.794189 64368 notify.go:220] Checking for updates...
I0815 18:25:53.796838 64368 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0815 18:25:53.798248 64368 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19450-13013/kubeconfig
I0815 18:25:53.799586 64368 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19450-13013/.minikube
I0815 18:25:53.800816 64368 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0815 18:25:53.802080 64368 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0815 18:25:53.803849 64368 config.go:182] Loaded profile config "cert-expiration-003860": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.0
I0815 18:25:53.803936 64368 config.go:182] Loaded profile config "kubernetes-upgrade-729203": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.0
I0815 18:25:53.804016 64368 config.go:182] Loaded profile config "stopped-upgrade-498665": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.24.1
I0815 18:25:53.804083 64368 driver.go:392] Setting default libvirt URI to qemu:///system
I0815 18:25:53.839002 64368 out.go:177] * Using the kvm2 driver based on user configuration
I0815 18:25:53.840361 64368 start.go:297] selected driver: kvm2
I0815 18:25:53.840381 64368 start.go:901] validating driver "kvm2" against <nil>
I0815 18:25:53.840392 64368 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0815 18:25:53.841058 64368 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 18:25:53.841172 64368 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19450-13013/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0815 18:25:53.855474 64368 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0815 18:25:53.855512 64368 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0815 18:25:53.855705 64368 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0815 18:25:53.855758 64368 cni.go:84] Creating CNI manager for ""
I0815 18:25:53.855770 64368 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0815 18:25:53.855781 64368 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0815 18:25:53.855825 64368 start.go:340] cluster config:
{Name:old-k8s-version-278865 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-278865 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 18:25:53.855912 64368 iso.go:125] acquiring lock: {Name:mk7679adb3d429c01d170a7f2d45922a687c8479 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 18:25:53.857536 64368 out.go:177] * Starting "old-k8s-version-278865" primary control-plane node in "old-k8s-version-278865" cluster
I0815 18:25:53.858613 64368 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0815 18:25:53.858644 64368 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0815 18:25:53.858654 64368 cache.go:56] Caching tarball of preloaded images
I0815 18:25:53.858717 64368 preload.go:172] Found /home/jenkins/minikube-integration/19450-13013/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0815 18:25:53.858726 64368 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0815 18:25:53.858801 64368 profile.go:143] Saving config to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/config.json ...
I0815 18:25:53.858816 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/config.json: {Name:mkf6d362ba564d7b775ae42e63037d279f71b39d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:25:53.858933 64368 start.go:360] acquireMachinesLock for old-k8s-version-278865: {Name:mk5c94326054b6faebe87b43653bce73979385d8 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0815 18:26:19.737924 64368 start.go:364] duration metric: took 25.878924414s to acquireMachinesLock for "old-k8s-version-278865"
I0815 18:26:19.738027 64368 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-278865 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19443/minikube-v1.33.1-1723650137-19443-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-278865 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0815 18:26:19.738173 64368 start.go:125] createHost starting for "" (driver="kvm2")
I0815 18:26:19.740156 64368 out.go:235] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0815 18:26:19.740389 64368 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0815 18:26:19.740440 64368 main.go:141] libmachine: Launching plugin server for driver kvm2
I0815 18:26:19.757943 64368 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:45725
I0815 18:26:19.758472 64368 main.go:141] libmachine: () Calling .GetVersion
I0815 18:26:19.759094 64368 main.go:141] libmachine: Using API Version 1
I0815 18:26:19.759120 64368 main.go:141] libmachine: () Calling .SetConfigRaw
I0815 18:26:19.759415 64368 main.go:141] libmachine: () Calling .GetMachineName
I0815 18:26:19.759589 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetMachineName
I0815 18:26:19.759716 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:19.759845 64368 start.go:159] libmachine.API.Create for "old-k8s-version-278865" (driver="kvm2")
I0815 18:26:19.759884 64368 client.go:168] LocalClient.Create starting
I0815 18:26:19.759929 64368 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem
I0815 18:26:19.759966 64368 main.go:141] libmachine: Decoding PEM data...
I0815 18:26:19.759987 64368 main.go:141] libmachine: Parsing certificate...
I0815 18:26:19.760051 64368 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19450-13013/.minikube/certs/cert.pem
I0815 18:26:19.760079 64368 main.go:141] libmachine: Decoding PEM data...
I0815 18:26:19.760095 64368 main.go:141] libmachine: Parsing certificate...
I0815 18:26:19.760130 64368 main.go:141] libmachine: Running pre-create checks...
I0815 18:26:19.760143 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .PreCreateCheck
I0815 18:26:19.760524 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetConfigRaw
I0815 18:26:19.760951 64368 main.go:141] libmachine: Creating machine...
I0815 18:26:19.760975 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .Create
I0815 18:26:19.761190 64368 main.go:141] libmachine: (old-k8s-version-278865) Creating KVM machine...
I0815 18:26:19.762390 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found existing default KVM network
I0815 18:26:19.764362 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:19.764177 64609 network.go:206] using free private subnet 192.168.39.0/24: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000014d20}
I0815 18:26:19.764390 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | created network xml:
I0815 18:26:19.764432 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <network>
I0815 18:26:19.764467 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <name>mk-old-k8s-version-278865</name>
I0815 18:26:19.764480 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <dns enable='no'/>
I0815 18:26:19.764504 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG |
I0815 18:26:19.764547 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <ip address='192.168.39.1' netmask='255.255.255.0'>
I0815 18:26:19.764569 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <dhcp>
I0815 18:26:19.764582 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | <range start='192.168.39.2' end='192.168.39.253'/>
I0815 18:26:19.764599 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | </dhcp>
I0815 18:26:19.764613 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | </ip>
I0815 18:26:19.764620 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG |
I0815 18:26:19.764629 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | </network>
I0815 18:26:19.764639 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG |
I0815 18:26:19.770133 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | trying to create private KVM network mk-old-k8s-version-278865 192.168.39.0/24...
I0815 18:26:19.840465 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | private KVM network mk-old-k8s-version-278865 192.168.39.0/24 created
I0815 18:26:19.840509 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:19.840444 64609 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19450-13013/.minikube
I0815 18:26:19.840523 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting up store path in /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865 ...
I0815 18:26:19.840541 64368 main.go:141] libmachine: (old-k8s-version-278865) Building disk image from file:///home/jenkins/minikube-integration/19450-13013/.minikube/cache/iso/amd64/minikube-v1.33.1-1723650137-19443-amd64.iso
I0815 18:26:19.840614 64368 main.go:141] libmachine: (old-k8s-version-278865) Downloading /home/jenkins/minikube-integration/19450-13013/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19450-13013/.minikube/cache/iso/amd64/minikube-v1.33.1-1723650137-19443-amd64.iso...
I0815 18:26:20.105974 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:20.105846 64609 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa...
I0815 18:26:20.292417 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:20.292267 64609 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/old-k8s-version-278865.rawdisk...
I0815 18:26:20.292451 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Writing magic tar header
I0815 18:26:20.292470 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Writing SSH key tar header
I0815 18:26:20.292484 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:20.292386 64609 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865 ...
I0815 18:26:20.292587 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865
I0815 18:26:20.292647 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19450-13013/.minikube/machines
I0815 18:26:20.292665 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865 (perms=drwx------)
I0815 18:26:20.292676 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19450-13013/.minikube
I0815 18:26:20.292690 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19450-13013
I0815 18:26:20.292705 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins/minikube-integration/19450-13013/.minikube/machines (perms=drwxr-xr-x)
I0815 18:26:20.292715 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0815 18:26:20.292728 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home/jenkins
I0815 18:26:20.292739 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Checking permissions on dir: /home
I0815 18:26:20.292767 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins/minikube-integration/19450-13013/.minikube (perms=drwxr-xr-x)
I0815 18:26:20.292795 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Skipping /home - not owner
I0815 18:26:20.292807 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins/minikube-integration/19450-13013 (perms=drwxrwxr-x)
I0815 18:26:20.292822 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0815 18:26:20.292835 64368 main.go:141] libmachine: (old-k8s-version-278865) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0815 18:26:20.292850 64368 main.go:141] libmachine: (old-k8s-version-278865) Creating domain...
I0815 18:26:20.294096 64368 main.go:141] libmachine: (old-k8s-version-278865) define libvirt domain using xml:
I0815 18:26:20.294114 64368 main.go:141] libmachine: (old-k8s-version-278865) <domain type='kvm'>
I0815 18:26:20.294124 64368 main.go:141] libmachine: (old-k8s-version-278865) <name>old-k8s-version-278865</name>
I0815 18:26:20.294132 64368 main.go:141] libmachine: (old-k8s-version-278865) <memory unit='MiB'>2200</memory>
I0815 18:26:20.294143 64368 main.go:141] libmachine: (old-k8s-version-278865) <vcpu>2</vcpu>
I0815 18:26:20.294154 64368 main.go:141] libmachine: (old-k8s-version-278865) <features>
I0815 18:26:20.294173 64368 main.go:141] libmachine: (old-k8s-version-278865) <acpi/>
I0815 18:26:20.294185 64368 main.go:141] libmachine: (old-k8s-version-278865) <apic/>
I0815 18:26:20.294194 64368 main.go:141] libmachine: (old-k8s-version-278865) <pae/>
I0815 18:26:20.294215 64368 main.go:141] libmachine: (old-k8s-version-278865)
I0815 18:26:20.294228 64368 main.go:141] libmachine: (old-k8s-version-278865) </features>
I0815 18:26:20.294239 64368 main.go:141] libmachine: (old-k8s-version-278865) <cpu mode='host-passthrough'>
I0815 18:26:20.294248 64368 main.go:141] libmachine: (old-k8s-version-278865)
I0815 18:26:20.294257 64368 main.go:141] libmachine: (old-k8s-version-278865) </cpu>
I0815 18:26:20.294287 64368 main.go:141] libmachine: (old-k8s-version-278865) <os>
I0815 18:26:20.294301 64368 main.go:141] libmachine: (old-k8s-version-278865) <type>hvm</type>
I0815 18:26:20.294314 64368 main.go:141] libmachine: (old-k8s-version-278865) <boot dev='cdrom'/>
I0815 18:26:20.294325 64368 main.go:141] libmachine: (old-k8s-version-278865) <boot dev='hd'/>
I0815 18:26:20.294337 64368 main.go:141] libmachine: (old-k8s-version-278865) <bootmenu enable='no'/>
I0815 18:26:20.294346 64368 main.go:141] libmachine: (old-k8s-version-278865) </os>
I0815 18:26:20.294355 64368 main.go:141] libmachine: (old-k8s-version-278865) <devices>
I0815 18:26:20.294373 64368 main.go:141] libmachine: (old-k8s-version-278865) <disk type='file' device='cdrom'>
I0815 18:26:20.294388 64368 main.go:141] libmachine: (old-k8s-version-278865) <source file='/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/boot2docker.iso'/>
I0815 18:26:20.294398 64368 main.go:141] libmachine: (old-k8s-version-278865) <target dev='hdc' bus='scsi'/>
I0815 18:26:20.294407 64368 main.go:141] libmachine: (old-k8s-version-278865) <readonly/>
I0815 18:26:20.294416 64368 main.go:141] libmachine: (old-k8s-version-278865) </disk>
I0815 18:26:20.294425 64368 main.go:141] libmachine: (old-k8s-version-278865) <disk type='file' device='disk'>
I0815 18:26:20.294438 64368 main.go:141] libmachine: (old-k8s-version-278865) <driver name='qemu' type='raw' cache='default' io='threads' />
I0815 18:26:20.294456 64368 main.go:141] libmachine: (old-k8s-version-278865) <source file='/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/old-k8s-version-278865.rawdisk'/>
I0815 18:26:20.294466 64368 main.go:141] libmachine: (old-k8s-version-278865) <target dev='hda' bus='virtio'/>
I0815 18:26:20.294479 64368 main.go:141] libmachine: (old-k8s-version-278865) </disk>
I0815 18:26:20.294494 64368 main.go:141] libmachine: (old-k8s-version-278865) <interface type='network'>
I0815 18:26:20.294506 64368 main.go:141] libmachine: (old-k8s-version-278865) <source network='mk-old-k8s-version-278865'/>
I0815 18:26:20.294517 64368 main.go:141] libmachine: (old-k8s-version-278865) <model type='virtio'/>
I0815 18:26:20.294549 64368 main.go:141] libmachine: (old-k8s-version-278865) </interface>
I0815 18:26:20.294565 64368 main.go:141] libmachine: (old-k8s-version-278865) <interface type='network'>
I0815 18:26:20.294579 64368 main.go:141] libmachine: (old-k8s-version-278865) <source network='default'/>
I0815 18:26:20.294589 64368 main.go:141] libmachine: (old-k8s-version-278865) <model type='virtio'/>
I0815 18:26:20.294600 64368 main.go:141] libmachine: (old-k8s-version-278865) </interface>
I0815 18:26:20.294611 64368 main.go:141] libmachine: (old-k8s-version-278865) <serial type='pty'>
I0815 18:26:20.294620 64368 main.go:141] libmachine: (old-k8s-version-278865) <target port='0'/>
I0815 18:26:20.294630 64368 main.go:141] libmachine: (old-k8s-version-278865) </serial>
I0815 18:26:20.294638 64368 main.go:141] libmachine: (old-k8s-version-278865) <console type='pty'>
I0815 18:26:20.294649 64368 main.go:141] libmachine: (old-k8s-version-278865) <target type='serial' port='0'/>
I0815 18:26:20.294660 64368 main.go:141] libmachine: (old-k8s-version-278865) </console>
I0815 18:26:20.294670 64368 main.go:141] libmachine: (old-k8s-version-278865) <rng model='virtio'>
I0815 18:26:20.294680 64368 main.go:141] libmachine: (old-k8s-version-278865) <backend model='random'>/dev/random</backend>
I0815 18:26:20.294690 64368 main.go:141] libmachine: (old-k8s-version-278865) </rng>
I0815 18:26:20.294698 64368 main.go:141] libmachine: (old-k8s-version-278865)
I0815 18:26:20.294708 64368 main.go:141] libmachine: (old-k8s-version-278865)
I0815 18:26:20.294716 64368 main.go:141] libmachine: (old-k8s-version-278865) </devices>
I0815 18:26:20.294725 64368 main.go:141] libmachine: (old-k8s-version-278865) </domain>
I0815 18:26:20.294736 64368 main.go:141] libmachine: (old-k8s-version-278865)
I0815 18:26:20.303255 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:43:52:45 in network default
I0815 18:26:20.304153 64368 main.go:141] libmachine: (old-k8s-version-278865) Ensuring networks are active...
I0815 18:26:20.304183 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:20.305178 64368 main.go:141] libmachine: (old-k8s-version-278865) Ensuring network default is active
I0815 18:26:20.305666 64368 main.go:141] libmachine: (old-k8s-version-278865) Ensuring network mk-old-k8s-version-278865 is active
I0815 18:26:20.306338 64368 main.go:141] libmachine: (old-k8s-version-278865) Getting domain xml...
I0815 18:26:20.307218 64368 main.go:141] libmachine: (old-k8s-version-278865) Creating domain...
I0815 18:26:21.786017 64368 main.go:141] libmachine: (old-k8s-version-278865) Waiting to get IP...
I0815 18:26:21.787235 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:21.787739 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:21.787766 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:21.787719 64609 retry.go:31] will retry after 215.934825ms: waiting for machine to come up
I0815 18:26:22.005367 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:22.006058 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:22.006083 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:22.005960 64609 retry.go:31] will retry after 352.288336ms: waiting for machine to come up
I0815 18:26:22.359585 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:22.360116 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:22.360151 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:22.360039 64609 retry.go:31] will retry after 339.814419ms: waiting for machine to come up
I0815 18:26:22.701729 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:22.702318 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:22.702340 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:22.702223 64609 retry.go:31] will retry after 480.397412ms: waiting for machine to come up
I0815 18:26:23.183922 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:23.184448 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:23.184476 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:23.184406 64609 retry.go:31] will retry after 591.567415ms: waiting for machine to come up
I0815 18:26:23.777218 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:23.777753 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:23.777799 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:23.777713 64609 retry.go:31] will retry after 855.993413ms: waiting for machine to come up
I0815 18:26:24.635176 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:24.635663 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:24.635706 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:24.635619 64609 retry.go:31] will retry after 989.25383ms: waiting for machine to come up
I0815 18:26:25.626811 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:25.627281 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:25.627304 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:25.627243 64609 retry.go:31] will retry after 1.243107723s: waiting for machine to come up
I0815 18:26:26.872518 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:26.872998 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:26.873024 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:26.872953 64609 retry.go:31] will retry after 1.446663155s: waiting for machine to come up
I0815 18:26:28.321430 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:28.321974 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:28.322003 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:28.321918 64609 retry.go:31] will retry after 1.709476148s: waiting for machine to come up
I0815 18:26:30.032912 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:30.033435 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:30.033463 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:30.033380 64609 retry.go:31] will retry after 2.467536976s: waiting for machine to come up
I0815 18:26:32.502319 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:32.502954 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:32.503016 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:32.502925 64609 retry.go:31] will retry after 2.261157367s: waiting for machine to come up
I0815 18:26:34.765746 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:34.766202 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:34.766222 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:34.766170 64609 retry.go:31] will retry after 4.21053318s: waiting for machine to come up
I0815 18:26:38.977877 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:38.978302 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find current IP address of domain old-k8s-version-278865 in network mk-old-k8s-version-278865
I0815 18:26:38.978327 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | I0815 18:26:38.978262 64609 retry.go:31] will retry after 3.959621261s: waiting for machine to come up
I0815 18:26:42.941583 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:42.942026 64368 main.go:141] libmachine: (old-k8s-version-278865) Found IP for machine: 192.168.39.89
I0815 18:26:42.942043 64368 main.go:141] libmachine: (old-k8s-version-278865) Reserving static IP address...
I0815 18:26:42.942057 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has current primary IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:42.942578 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-278865", mac: "52:54:00:b7:18:0a", ip: "192.168.39.89"} in network mk-old-k8s-version-278865
I0815 18:26:43.020225 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Getting to WaitForSSH function...
I0815 18:26:43.020252 64368 main.go:141] libmachine: (old-k8s-version-278865) Reserved static IP address: 192.168.39.89
I0815 18:26:43.020265 64368 main.go:141] libmachine: (old-k8s-version-278865) Waiting for SSH to be available...
I0815 18:26:43.023222 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.023781 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:minikube Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.023806 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.024005 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Using SSH client type: external
I0815 18:26:43.024046 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Using SSH private key: /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa (-rw-------)
I0815 18:26:43.024095 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.89 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa -p 22] /usr/bin/ssh <nil>}
I0815 18:26:43.024110 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | About to run SSH command:
I0815 18:26:43.024130 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | exit 0
I0815 18:26:43.148744 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | SSH cmd err, output: <nil>:
I0815 18:26:43.148999 64368 main.go:141] libmachine: (old-k8s-version-278865) KVM machine creation complete!
I0815 18:26:43.149424 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetConfigRaw
I0815 18:26:43.149994 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:43.150217 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:43.150431 64368 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0815 18:26:43.150455 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetState
I0815 18:26:43.151866 64368 main.go:141] libmachine: Detecting operating system of created instance...
I0815 18:26:43.151888 64368 main.go:141] libmachine: Waiting for SSH to be available...
I0815 18:26:43.151895 64368 main.go:141] libmachine: Getting to WaitForSSH function...
I0815 18:26:43.151904 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:43.154259 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.154571 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.154603 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.154773 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:43.154949 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.155125 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.155271 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:43.155445 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:43.155728 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:43.155747 64368 main.go:141] libmachine: About to run SSH command:
exit 0
I0815 18:26:43.260226 64368 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0815 18:26:43.260249 64368 main.go:141] libmachine: Detecting the provisioner...
I0815 18:26:43.260257 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:43.263094 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.263460 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.263489 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.263653 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:43.263823 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.264103 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.264297 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:43.264463 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:43.264665 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:43.264680 64368 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0815 18:26:43.369481 64368 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0815 18:26:43.369606 64368 main.go:141] libmachine: found compatible host: buildroot
I0815 18:26:43.369619 64368 main.go:141] libmachine: Provisioning with buildroot...
I0815 18:26:43.369631 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetMachineName
I0815 18:26:43.369858 64368 buildroot.go:166] provisioning hostname "old-k8s-version-278865"
I0815 18:26:43.369885 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetMachineName
I0815 18:26:43.370058 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:43.372749 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.373121 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.373148 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.373271 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:43.373431 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.373535 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.373632 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:43.373819 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:43.373984 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:43.373996 64368 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-278865 && echo "old-k8s-version-278865" | sudo tee /etc/hostname
I0815 18:26:43.495620 64368 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-278865
I0815 18:26:43.495643 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:43.498687 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.499012 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.499051 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.499196 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:43.499359 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.499523 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:43.499658 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:43.499819 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:43.500043 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:43.500070 64368 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-278865' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-278865/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-278865' | sudo tee -a /etc/hosts;
fi
fi
I0815 18:26:43.613817 64368 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0815 18:26:43.613850 64368 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19450-13013/.minikube CaCertPath:/home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19450-13013/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19450-13013/.minikube}
I0815 18:26:43.613884 64368 buildroot.go:174] setting up certificates
I0815 18:26:43.613894 64368 provision.go:84] configureAuth start
I0815 18:26:43.613912 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetMachineName
I0815 18:26:43.614152 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetIP
I0815 18:26:43.616903 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.617304 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.617338 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.617471 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:43.619662 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.619962 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:43.619999 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:43.620141 64368 provision.go:143] copyHostCerts
I0815 18:26:43.620197 64368 exec_runner.go:144] found /home/jenkins/minikube-integration/19450-13013/.minikube/ca.pem, removing ...
I0815 18:26:43.620215 64368 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19450-13013/.minikube/ca.pem
I0815 18:26:43.620273 64368 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19450-13013/.minikube/ca.pem (1082 bytes)
I0815 18:26:43.620445 64368 exec_runner.go:144] found /home/jenkins/minikube-integration/19450-13013/.minikube/cert.pem, removing ...
I0815 18:26:43.620455 64368 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19450-13013/.minikube/cert.pem
I0815 18:26:43.620479 64368 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19450-13013/.minikube/cert.pem (1123 bytes)
I0815 18:26:43.620582 64368 exec_runner.go:144] found /home/jenkins/minikube-integration/19450-13013/.minikube/key.pem, removing ...
I0815 18:26:43.620590 64368 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19450-13013/.minikube/key.pem
I0815 18:26:43.620609 64368 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19450-13013/.minikube/key.pem (1675 bytes)
I0815 18:26:43.620668 64368 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19450-13013/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-278865 san=[127.0.0.1 192.168.39.89 localhost minikube old-k8s-version-278865]
I0815 18:26:44.187517 64368 provision.go:177] copyRemoteCerts
I0815 18:26:44.187575 64368 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0815 18:26:44.187598 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.189998 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.190293 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.190322 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.190466 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.190712 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.190904 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.191064 64368 sshutil.go:53] new ssh client: &{IP:192.168.39.89 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa Username:docker}
I0815 18:26:44.275738 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0815 18:26:44.298801 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0815 18:26:44.322708 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0815 18:26:44.345664 64368 provision.go:87] duration metric: took 731.75755ms to configureAuth
I0815 18:26:44.345693 64368 buildroot.go:189] setting minikube options for container-runtime
I0815 18:26:44.345879 64368 config.go:182] Loaded profile config "old-k8s-version-278865": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0815 18:26:44.345951 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.348519 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.348878 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.348897 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.349049 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.349241 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.349411 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.349536 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.349711 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:44.349910 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:44.349934 64368 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0815 18:26:44.618371 64368 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0815 18:26:44.618399 64368 main.go:141] libmachine: Checking connection to Docker...
I0815 18:26:44.618408 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetURL
I0815 18:26:44.619722 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | Using libvirt version 6000000
I0815 18:26:44.621759 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.622141 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.622173 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.622278 64368 main.go:141] libmachine: Docker is up and running!
I0815 18:26:44.622299 64368 main.go:141] libmachine: Reticulating splines...
I0815 18:26:44.622306 64368 client.go:171] duration metric: took 24.862411526s to LocalClient.Create
I0815 18:26:44.622336 64368 start.go:167] duration metric: took 24.862501737s to libmachine.API.Create "old-k8s-version-278865"
I0815 18:26:44.622345 64368 start.go:293] postStartSetup for "old-k8s-version-278865" (driver="kvm2")
I0815 18:26:44.622354 64368 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0815 18:26:44.622372 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:44.622625 64368 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0815 18:26:44.622656 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.624769 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.625099 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.625126 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.625269 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.625451 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.625624 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.625791 64368 sshutil.go:53] new ssh client: &{IP:192.168.39.89 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa Username:docker}
I0815 18:26:44.707742 64368 ssh_runner.go:195] Run: cat /etc/os-release
I0815 18:26:44.711983 64368 info.go:137] Remote host: Buildroot 2023.02.9
I0815 18:26:44.712010 64368 filesync.go:126] Scanning /home/jenkins/minikube-integration/19450-13013/.minikube/addons for local assets ...
I0815 18:26:44.712082 64368 filesync.go:126] Scanning /home/jenkins/minikube-integration/19450-13013/.minikube/files for local assets ...
I0815 18:26:44.712189 64368 filesync.go:149] local asset: /home/jenkins/minikube-integration/19450-13013/.minikube/files/etc/ssl/certs/202192.pem -> 202192.pem in /etc/ssl/certs
I0815 18:26:44.712278 64368 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0815 18:26:44.721962 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/files/etc/ssl/certs/202192.pem --> /etc/ssl/certs/202192.pem (1708 bytes)
I0815 18:26:44.745826 64368 start.go:296] duration metric: took 123.470495ms for postStartSetup
I0815 18:26:44.745872 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetConfigRaw
I0815 18:26:44.746401 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetIP
I0815 18:26:44.748801 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.749223 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.749245 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.749591 64368 profile.go:143] Saving config to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/config.json ...
I0815 18:26:44.749795 64368 start.go:128] duration metric: took 25.011607097s to createHost
I0815 18:26:44.749819 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.752199 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.752643 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.752667 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.752833 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.753037 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.753188 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.753331 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.753492 64368 main.go:141] libmachine: Using SSH client type: native
I0815 18:26:44.753656 64368 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82f9c0] 0x832720 <nil> [] 0s} 192.168.39.89 22 <nil> <nil>}
I0815 18:26:44.753675 64368 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0815 18:26:44.857401 64368 main.go:141] libmachine: SSH cmd err, output: <nil>: 1723746404.831725234
I0815 18:26:44.857433 64368 fix.go:216] guest clock: 1723746404.831725234
I0815 18:26:44.857444 64368 fix.go:229] Guest: 2024-08-15 18:26:44.831725234 +0000 UTC Remote: 2024-08-15 18:26:44.749808719 +0000 UTC m=+50.995480451 (delta=81.916515ms)
I0815 18:26:44.857483 64368 fix.go:200] guest clock delta is within tolerance: 81.916515ms
I0815 18:26:44.857491 64368 start.go:83] releasing machines lock for "old-k8s-version-278865", held for 25.119510908s
I0815 18:26:44.857518 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:44.857805 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetIP
I0815 18:26:44.860347 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.860781 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.860810 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.860957 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:44.861677 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:44.861892 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .DriverName
I0815 18:26:44.861979 64368 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0815 18:26:44.862025 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.862102 64368 ssh_runner.go:195] Run: cat /version.json
I0815 18:26:44.862125 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHHostname
I0815 18:26:44.865296 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.865461 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.865682 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.865711 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.865863 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.865878 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:44.865906 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:44.866041 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.866055 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHPort
I0815 18:26:44.866211 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHKeyPath
I0815 18:26:44.866215 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.866382 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetSSHUsername
I0815 18:26:44.866383 64368 sshutil.go:53] new ssh client: &{IP:192.168.39.89 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa Username:docker}
I0815 18:26:44.866560 64368 sshutil.go:53] new ssh client: &{IP:192.168.39.89 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19450-13013/.minikube/machines/old-k8s-version-278865/id_rsa Username:docker}
I0815 18:26:44.968627 64368 ssh_runner.go:195] Run: systemctl --version
I0815 18:26:44.974805 64368 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0815 18:26:45.138147 64368 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0815 18:26:45.144313 64368 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0815 18:26:45.144377 64368 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0815 18:26:45.160056 64368 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0815 18:26:45.160081 64368 start.go:495] detecting cgroup driver to use...
I0815 18:26:45.160158 64368 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0815 18:26:45.178269 64368 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0815 18:26:45.193028 64368 docker.go:217] disabling cri-docker service (if available) ...
I0815 18:26:45.193089 64368 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0815 18:26:45.207037 64368 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0815 18:26:45.226663 64368 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0815 18:26:45.358346 64368 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0815 18:26:45.506003 64368 docker.go:233] disabling docker service ...
I0815 18:26:45.506076 64368 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0815 18:26:45.527696 64368 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0815 18:26:45.552024 64368 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0815 18:26:45.701984 64368 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0815 18:26:45.817868 64368 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0815 18:26:45.832805 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0815 18:26:45.854229 64368 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0815 18:26:45.854291 64368 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0815 18:26:45.866467 64368 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0815 18:26:45.866524 64368 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0815 18:26:45.877712 64368 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0815 18:26:45.890109 64368 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0815 18:26:45.902120 64368 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0815 18:26:45.914157 64368 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0815 18:26:45.925673 64368 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0815 18:26:45.925731 64368 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0815 18:26:45.946835 64368 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0815 18:26:45.958098 64368 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 18:26:46.099956 64368 ssh_runner.go:195] Run: sudo systemctl restart crio
I0815 18:26:46.255366 64368 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0815 18:26:46.255441 64368 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0815 18:26:46.260549 64368 start.go:563] Will wait 60s for crictl version
I0815 18:26:46.260611 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:46.264202 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0815 18:26:46.308555 64368 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0815 18:26:46.308649 64368 ssh_runner.go:195] Run: crio --version
I0815 18:26:46.337148 64368 ssh_runner.go:195] Run: crio --version
I0815 18:26:46.367888 64368 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0815 18:26:46.369308 64368 main.go:141] libmachine: (old-k8s-version-278865) Calling .GetIP
I0815 18:26:46.372288 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:46.372697 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:b7:18:0a", ip: ""} in network mk-old-k8s-version-278865: {Iface:virbr4 ExpiryTime:2024-08-15 19:26:35 +0000 UTC Type:0 Mac:52:54:00:b7:18:0a Iaid: IPaddr:192.168.39.89 Prefix:24 Hostname:old-k8s-version-278865 Clientid:01:52:54:00:b7:18:0a}
I0815 18:26:46.372730 64368 main.go:141] libmachine: (old-k8s-version-278865) DBG | domain old-k8s-version-278865 has defined IP address 192.168.39.89 and MAC address 52:54:00:b7:18:0a in network mk-old-k8s-version-278865
I0815 18:26:46.372922 64368 ssh_runner.go:195] Run: grep 192.168.39.1 host.minikube.internal$ /etc/hosts
I0815 18:26:46.377064 64368 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.39.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 18:26:46.391250 64368 kubeadm.go:883] updating cluster {Name:old-k8s-version-278865 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19443/minikube-v1.33.1-1723650137-19443-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-278865 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.89 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0815 18:26:46.391386 64368 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0815 18:26:46.391451 64368 ssh_runner.go:195] Run: sudo crictl images --output json
I0815 18:26:46.433479 64368 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0815 18:26:46.433561 64368 ssh_runner.go:195] Run: which lz4
I0815 18:26:46.438432 64368 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0815 18:26:46.442642 64368 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0815 18:26:46.442666 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0815 18:26:48.132589 64368 crio.go:462] duration metric: took 1.694196037s to copy over tarball
I0815 18:26:48.132674 64368 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0815 18:26:50.690623 64368 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.557914585s)
I0815 18:26:50.690667 64368 crio.go:469] duration metric: took 2.558038185s to extract the tarball
I0815 18:26:50.690678 64368 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0815 18:26:50.734827 64368 ssh_runner.go:195] Run: sudo crictl images --output json
I0815 18:26:50.781327 64368 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0815 18:26:50.781351 64368 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0815 18:26:50.781472 64368 image.go:135] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:50.781494 64368 image.go:135] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:50.781509 64368 image.go:135] retrieving image: registry.k8s.io/coredns:1.7.0
I0815 18:26:50.781522 64368 image.go:135] retrieving image: registry.k8s.io/pause:3.2
I0815 18:26:50.781438 64368 image.go:135] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:50.781548 64368 image.go:135] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:50.781438 64368 image.go:135] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0815 18:26:50.781440 64368 image.go:135] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0815 18:26:50.783531 64368 image.go:178] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0815 18:26:50.783595 64368 image.go:178] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:50.783606 64368 image.go:178] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:50.783621 64368 image.go:178] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:50.783542 64368 image.go:178] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0815 18:26:50.783549 64368 image.go:178] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0815 18:26:50.783676 64368 image.go:178] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0815 18:26:50.783567 64368 image.go:178] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:50.949780 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:50.960111 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0815 18:26:51.000741 64368 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0815 18:26:51.000796 64368 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:51.000846 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.008063 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0815 18:26:51.012373 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:51.012446 64368 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0815 18:26:51.012518 64368 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0815 18:26:51.012561 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.059855 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0815 18:26:51.059964 64368 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0815 18:26:51.060036 64368 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0815 18:26:51.060080 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:51.060116 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.097244 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0815 18:26:51.113511 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0815 18:26:51.113550 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0815 18:26:51.113604 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0815 18:26:51.137186 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:51.155409 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:51.173814 64368 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0815 18:26:51.173862 64368 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0815 18:26:51.173912 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.228022 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0815 18:26:51.238588 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0815 18:26:51.238597 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0815 18:26:51.278941 64368 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0815 18:26:51.278981 64368 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:51.279032 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.283071 64368 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0815 18:26:51.283118 64368 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:51.283172 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.283177 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0815 18:26:51.307891 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0815 18:26:51.320952 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0815 18:26:51.321007 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:51.321008 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:51.325798 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:51.353707 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0815 18:26:51.416632 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0815 18:26:51.435034 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:51.435089 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:51.459584 64368 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0815 18:26:51.459633 64368 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:51.459664 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0815 18:26:51.459679 64368 ssh_runner.go:195] Run: which crictl
I0815 18:26:51.513398 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0815 18:26:51.513440 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0815 18:26:51.530000 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:51.530050 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0815 18:26:51.586456 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0815 18:26:51.586462 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0815 18:26:51.597704 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:51.629930 64368 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0815 18:26:51.674431 64368 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0815 18:26:51.711237 64368 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0815 18:26:51.863355 64368 cache_images.go:92] duration metric: took 1.081984361s to LoadCachedImages
W0815 18:26:51.863460 64368 out.go:270] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19450-13013/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0: no such file or directory
I0815 18:26:51.863476 64368 kubeadm.go:934] updating node { 192.168.39.89 8443 v1.20.0 crio true true} ...
I0815 18:26:51.863612 64368 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-278865 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.39.89
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-278865 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0815 18:26:51.863694 64368 ssh_runner.go:195] Run: crio config
I0815 18:26:51.915393 64368 cni.go:84] Creating CNI manager for ""
I0815 18:26:51.915466 64368 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0815 18:26:51.915482 64368 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0815 18:26:51.915509 64368 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.89 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-278865 NodeName:old-k8s-version-278865 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.89"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.39.89 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0815 18:26:51.915688 64368 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.39.89
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-278865"
kubeletExtraArgs:
node-ip: 192.168.39.89
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.39.89"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0815 18:26:51.915758 64368 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0815 18:26:51.930339 64368 binaries.go:44] Found k8s binaries, skipping transfer
I0815 18:26:51.930423 64368 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0815 18:26:51.943602 64368 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0815 18:26:51.960687 64368 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0815 18:26:51.978511 64368 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0815 18:26:51.996372 64368 ssh_runner.go:195] Run: grep 192.168.39.89 control-plane.minikube.internal$ /etc/hosts
I0815 18:26:52.000262 64368 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.39.89 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 18:26:52.013040 64368 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 18:26:52.138671 64368 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0815 18:26:52.157474 64368 certs.go:68] Setting up /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865 for IP: 192.168.39.89
I0815 18:26:52.157508 64368 certs.go:194] generating shared ca certs ...
I0815 18:26:52.157530 64368 certs.go:226] acquiring lock for ca certs: {Name:mkaf2a49c545ba9ac79ccda0cd19bc293b18915b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.157717 64368 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19450-13013/.minikube/ca.key
I0815 18:26:52.157775 64368 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19450-13013/.minikube/proxy-client-ca.key
I0815 18:26:52.157793 64368 certs.go:256] generating profile certs ...
I0815 18:26:52.157870 64368 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.key
I0815 18:26:52.157891 64368 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.crt with IP's: []
I0815 18:26:52.256783 64368 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.crt ...
I0815 18:26:52.256817 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.crt: {Name:mk489eb0952cf53a915129fd288ab2fd07350a45 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.257013 64368 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.key ...
I0815 18:26:52.257029 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/client.key: {Name:mke75b69e3e7b80a3685923312134ea2bd16478b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.257133 64368 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key.b00e3c1a
I0815 18:26:52.257157 64368 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt.b00e3c1a with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.39.89]
I0815 18:26:52.514942 64368 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt.b00e3c1a ...
I0815 18:26:52.514971 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt.b00e3c1a: {Name:mk71bef92184d414517f936910c6b02a23ca09b2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.515125 64368 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key.b00e3c1a ...
I0815 18:26:52.515138 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key.b00e3c1a: {Name:mk37306c17621e8d5ca942be7928f51bd17080bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.515212 64368 certs.go:381] copying /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt.b00e3c1a -> /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt
I0815 18:26:52.515294 64368 certs.go:385] copying /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key.b00e3c1a -> /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key
I0815 18:26:52.515345 64368 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.key
I0815 18:26:52.515360 64368 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.crt with IP's: []
I0815 18:26:52.594313 64368 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.crt ...
I0815 18:26:52.594340 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.crt: {Name:mk9a46182f3609e6c7e843c3472924b6ae54f09a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.594530 64368 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.key ...
I0815 18:26:52.594547 64368 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.key: {Name:mk88cd43667643e2f89a51eb09f9690b55733f30 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 18:26:52.594747 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/20219.pem (1338 bytes)
W0815 18:26:52.594785 64368 certs.go:480] ignoring /home/jenkins/minikube-integration/19450-13013/.minikube/certs/20219_empty.pem, impossibly tiny 0 bytes
I0815 18:26:52.594795 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca-key.pem (1675 bytes)
I0815 18:26:52.594822 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/ca.pem (1082 bytes)
I0815 18:26:52.594844 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/cert.pem (1123 bytes)
I0815 18:26:52.594866 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/certs/key.pem (1675 bytes)
I0815 18:26:52.594902 64368 certs.go:484] found cert: /home/jenkins/minikube-integration/19450-13013/.minikube/files/etc/ssl/certs/202192.pem (1708 bytes)
I0815 18:26:52.595486 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0815 18:26:52.628555 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0815 18:26:52.656996 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0815 18:26:52.685663 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0815 18:26:52.710069 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0815 18:26:52.735575 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0815 18:26:52.759775 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0815 18:26:52.785418 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/profiles/old-k8s-version-278865/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0815 18:26:52.809846 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/files/etc/ssl/certs/202192.pem --> /usr/share/ca-certificates/202192.pem (1708 bytes)
I0815 18:26:52.836180 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0815 18:26:52.862384 64368 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19450-13013/.minikube/certs/20219.pem --> /usr/share/ca-certificates/20219.pem (1338 bytes)
I0815 18:26:52.890166 64368 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0815 18:26:52.908973 64368 ssh_runner.go:195] Run: openssl version
I0815 18:26:52.915186 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/202192.pem && ln -fs /usr/share/ca-certificates/202192.pem /etc/ssl/certs/202192.pem"
I0815 18:26:52.927217 64368 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/202192.pem
I0815 18:26:52.932155 64368 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 15 17:19 /usr/share/ca-certificates/202192.pem
I0815 18:26:52.932220 64368 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/202192.pem
I0815 18:26:52.938718 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/202192.pem /etc/ssl/certs/3ec20f2e.0"
I0815 18:26:52.950738 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0815 18:26:52.962656 64368 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0815 18:26:52.967654 64368 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 15 17:06 /usr/share/ca-certificates/minikubeCA.pem
I0815 18:26:52.967723 64368 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0815 18:26:52.974702 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0815 18:26:52.993677 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/20219.pem && ln -fs /usr/share/ca-certificates/20219.pem /etc/ssl/certs/20219.pem"
I0815 18:26:53.013657 64368 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/20219.pem
I0815 18:26:53.019869 64368 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 15 17:19 /usr/share/ca-certificates/20219.pem
I0815 18:26:53.019936 64368 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/20219.pem
I0815 18:26:53.030433 64368 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/20219.pem /etc/ssl/certs/51391683.0"
I0815 18:26:53.053149 64368 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0815 18:26:53.058133 64368 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0815 18:26:53.058206 64368 kubeadm.go:392] StartCluster: {Name:old-k8s-version-278865 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19443/minikube-v1.33.1-1723650137-19443-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-278865 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.89 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 18:26:53.058320 64368 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0815 18:26:53.058382 64368 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0815 18:26:53.102549 64368 cri.go:89] found id: ""
I0815 18:26:53.102635 64368 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0815 18:26:53.113775 64368 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0815 18:26:53.124452 64368 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0815 18:26:53.135251 64368 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0815 18:26:53.135277 64368 kubeadm.go:157] found existing configuration files:
I0815 18:26:53.135332 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0815 18:26:53.145809 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0815 18:26:53.145875 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0815 18:26:53.155995 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0815 18:26:53.166137 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0815 18:26:53.166212 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0815 18:26:53.176057 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0815 18:26:53.187552 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0815 18:26:53.187628 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0815 18:26:53.199467 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0815 18:26:53.209707 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0815 18:26:53.209774 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0815 18:26:53.220118 64368 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0815 18:26:53.362998 64368 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0815 18:26:53.363125 64368 kubeadm.go:310] [preflight] Running pre-flight checks
I0815 18:26:53.532702 64368 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0815 18:26:53.532876 64368 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0815 18:26:53.532987 64368 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0815 18:26:53.733641 64368 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0815 18:26:53.923997 64368 out.go:235] - Generating certificates and keys ...
I0815 18:26:53.924126 64368 kubeadm.go:310] [certs] Using existing ca certificate authority
I0815 18:26:53.924258 64368 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0815 18:26:53.945246 64368 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0815 18:26:54.037805 64368 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0815 18:26:54.212969 64368 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0815 18:26:54.386613 64368 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0815 18:26:54.622292 64368 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0815 18:26:54.622494 64368 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
I0815 18:26:54.764357 64368 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0815 18:26:54.764788 64368 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
I0815 18:26:54.952589 64368 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0815 18:26:55.225522 64368 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0815 18:26:55.650538 64368 kubeadm.go:310] [certs] Generating "sa" key and public key
I0815 18:26:55.650961 64368 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0815 18:26:55.762725 64368 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0815 18:26:56.074128 64368 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0815 18:26:56.262699 64368 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0815 18:26:56.452684 64368 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0815 18:26:56.468703 64368 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0815 18:26:56.469284 64368 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0815 18:26:56.469353 64368 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0815 18:26:56.630461 64368 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0815 18:26:56.632158 64368 out.go:235] - Booting up control plane ...
I0815 18:26:56.632301 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0815 18:26:56.640533 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0815 18:26:56.642342 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0815 18:26:56.644109 64368 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0815 18:26:56.650394 64368 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0815 18:27:36.643293 64368 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0815 18:27:36.643859 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:27:36.644100 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:27:41.644510 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:27:41.644794 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:27:51.644192 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:27:51.644439 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:28:11.643749 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:28:11.644003 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:28:51.645867 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:28:51.646502 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:28:51.646532 64368 kubeadm.go:310]
I0815 18:28:51.646620 64368 kubeadm.go:310] Unfortunately, an error has occurred:
I0815 18:28:51.646710 64368 kubeadm.go:310] timed out waiting for the condition
I0815 18:28:51.646720 64368 kubeadm.go:310]
I0815 18:28:51.646797 64368 kubeadm.go:310] This error is likely caused by:
I0815 18:28:51.646872 64368 kubeadm.go:310] - The kubelet is not running
I0815 18:28:51.647112 64368 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0815 18:28:51.647123 64368 kubeadm.go:310]
I0815 18:28:51.647356 64368 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0815 18:28:51.647436 64368 kubeadm.go:310] - 'systemctl status kubelet'
I0815 18:28:51.647509 64368 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0815 18:28:51.647521 64368 kubeadm.go:310]
I0815 18:28:51.647758 64368 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0815 18:28:51.647944 64368 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0815 18:28:51.647956 64368 kubeadm.go:310]
I0815 18:28:51.648183 64368 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0815 18:28:51.648389 64368 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0815 18:28:51.648576 64368 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0815 18:28:51.648817 64368 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0815 18:28:51.648839 64368 kubeadm.go:310]
I0815 18:28:51.649713 64368 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0815 18:28:51.649843 64368 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0815 18:28:51.649960 64368 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
W0815 18:28:51.650092 64368 out.go:270] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-278865] and IPs [192.168.39.89 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0815 18:28:51.650135 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0815 18:28:53.070008 64368 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force": (1.419845953s)
I0815 18:28:53.070101 64368 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0815 18:28:53.084149 64368 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0815 18:28:53.094153 64368 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0815 18:28:53.094170 64368 kubeadm.go:157] found existing configuration files:
I0815 18:28:53.094212 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0815 18:28:53.103880 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0815 18:28:53.103946 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0815 18:28:53.114099 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0815 18:28:53.123508 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0815 18:28:53.123574 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0815 18:28:53.133572 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0815 18:28:53.142595 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0815 18:28:53.142648 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0815 18:28:53.152725 64368 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0815 18:28:53.162196 64368 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0815 18:28:53.162253 64368 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0815 18:28:53.171521 64368 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0815 18:28:53.249366 64368 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0815 18:28:53.257406 64368 kubeadm.go:310] [preflight] Running pre-flight checks
I0815 18:28:53.413220 64368 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0815 18:28:53.413343 64368 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0815 18:28:53.413515 64368 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0815 18:28:53.612333 64368 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0815 18:28:53.614435 64368 out.go:235] - Generating certificates and keys ...
I0815 18:28:53.614548 64368 kubeadm.go:310] [certs] Using existing ca certificate authority
I0815 18:28:53.614635 64368 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0815 18:28:53.614765 64368 kubeadm.go:310] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0815 18:28:53.614853 64368 kubeadm.go:310] [certs] Using existing front-proxy-ca certificate authority
I0815 18:28:53.614947 64368 kubeadm.go:310] [certs] Using existing front-proxy-client certificate and key on disk
I0815 18:28:53.615019 64368 kubeadm.go:310] [certs] Using existing etcd/ca certificate authority
I0815 18:28:53.615325 64368 kubeadm.go:310] [certs] Using existing etcd/server certificate and key on disk
I0815 18:28:53.615699 64368 kubeadm.go:310] [certs] Using existing etcd/peer certificate and key on disk
I0815 18:28:53.616149 64368 kubeadm.go:310] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0815 18:28:53.616868 64368 kubeadm.go:310] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0815 18:28:53.616982 64368 kubeadm.go:310] [certs] Using the existing "sa" key
I0815 18:28:53.617071 64368 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0815 18:28:53.778844 64368 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0815 18:28:54.224573 64368 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0815 18:28:54.861636 64368 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0815 18:28:54.936684 64368 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0815 18:28:54.951036 64368 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0815 18:28:54.952440 64368 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0815 18:28:54.952544 64368 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0815 18:28:55.093981 64368 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0815 18:28:55.095841 64368 out.go:235] - Booting up control plane ...
I0815 18:28:55.095978 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0815 18:28:55.102399 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0815 18:28:55.102848 64368 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0815 18:28:55.105802 64368 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0815 18:28:55.108854 64368 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0815 18:29:35.111590 64368 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0815 18:29:35.111727 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:29:35.111951 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:29:40.112442 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:29:40.112658 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:29:50.113211 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:29:50.113425 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:30:10.112760 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:30:10.112953 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:30:50.113290 64368 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0815 18:30:50.113533 64368 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0815 18:30:50.113549 64368 kubeadm.go:310]
I0815 18:30:50.113608 64368 kubeadm.go:310] Unfortunately, an error has occurred:
I0815 18:30:50.113673 64368 kubeadm.go:310] timed out waiting for the condition
I0815 18:30:50.113684 64368 kubeadm.go:310]
I0815 18:30:50.113712 64368 kubeadm.go:310] This error is likely caused by:
I0815 18:30:50.113741 64368 kubeadm.go:310] - The kubelet is not running
I0815 18:30:50.113833 64368 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0815 18:30:50.113857 64368 kubeadm.go:310]
I0815 18:30:50.113996 64368 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0815 18:30:50.114049 64368 kubeadm.go:310] - 'systemctl status kubelet'
I0815 18:30:50.114095 64368 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0815 18:30:50.114105 64368 kubeadm.go:310]
I0815 18:30:50.114249 64368 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0815 18:30:50.114321 64368 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0815 18:30:50.114328 64368 kubeadm.go:310]
I0815 18:30:50.114426 64368 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0815 18:30:50.114547 64368 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0815 18:30:50.114663 64368 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0815 18:30:50.114764 64368 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0815 18:30:50.114773 64368 kubeadm.go:310]
I0815 18:30:50.115378 64368 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0815 18:30:50.115454 64368 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0815 18:30:50.115524 64368 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0815 18:30:50.115614 64368 kubeadm.go:394] duration metric: took 3m57.057412402s to StartCluster
I0815 18:30:50.115671 64368 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0815 18:30:50.115753 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 18:30:50.156963 64368 cri.go:89] found id: ""
I0815 18:30:50.157011 64368 logs.go:276] 0 containers: []
W0815 18:30:50.157023 64368 logs.go:278] No container was found matching "kube-apiserver"
I0815 18:30:50.157036 64368 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0815 18:30:50.157099 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 18:30:50.191959 64368 cri.go:89] found id: ""
I0815 18:30:50.191993 64368 logs.go:276] 0 containers: []
W0815 18:30:50.192004 64368 logs.go:278] No container was found matching "etcd"
I0815 18:30:50.192012 64368 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0815 18:30:50.192072 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 18:30:50.225988 64368 cri.go:89] found id: ""
I0815 18:30:50.226022 64368 logs.go:276] 0 containers: []
W0815 18:30:50.226033 64368 logs.go:278] No container was found matching "coredns"
I0815 18:30:50.226044 64368 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0815 18:30:50.226115 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 18:30:50.285980 64368 cri.go:89] found id: ""
I0815 18:30:50.286011 64368 logs.go:276] 0 containers: []
W0815 18:30:50.286025 64368 logs.go:278] No container was found matching "kube-scheduler"
I0815 18:30:50.286038 64368 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0815 18:30:50.286148 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 18:30:50.319265 64368 cri.go:89] found id: ""
I0815 18:30:50.319292 64368 logs.go:276] 0 containers: []
W0815 18:30:50.319303 64368 logs.go:278] No container was found matching "kube-proxy"
I0815 18:30:50.319311 64368 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0815 18:30:50.319373 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 18:30:50.352445 64368 cri.go:89] found id: ""
I0815 18:30:50.352471 64368 logs.go:276] 0 containers: []
W0815 18:30:50.352479 64368 logs.go:278] No container was found matching "kube-controller-manager"
I0815 18:30:50.352495 64368 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0815 18:30:50.352553 64368 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 18:30:50.389616 64368 cri.go:89] found id: ""
I0815 18:30:50.389646 64368 logs.go:276] 0 containers: []
W0815 18:30:50.389655 64368 logs.go:278] No container was found matching "kindnet"
I0815 18:30:50.389665 64368 logs.go:123] Gathering logs for dmesg ...
I0815 18:30:50.389677 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 18:30:50.405546 64368 logs.go:123] Gathering logs for describe nodes ...
I0815 18:30:50.405577 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0815 18:30:50.523336 64368 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0815 18:30:50.523364 64368 logs.go:123] Gathering logs for CRI-O ...
I0815 18:30:50.523379 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0815 18:30:50.631854 64368 logs.go:123] Gathering logs for container status ...
I0815 18:30:50.631891 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 18:30:50.670964 64368 logs.go:123] Gathering logs for kubelet ...
I0815 18:30:50.670996 64368 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 18:30:50.723714 64368 out.go:418] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0815 18:30:50.723773 64368 out.go:270] *
*
W0815 18:30:50.723842 64368 out.go:270] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0815 18:30:50.723862 64368 out.go:270] *
*
W0815 18:30:50.724680 64368 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0815 18:30:50.727644 64368 out.go:201]
W0815 18:30:50.729178 64368 out.go:270] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0815 18:30:50.729223 64368 out.go:270] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0815 18:30:50.729246 64368 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0815 18:30:50.730706 64368 out.go:201]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-278865 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-278865 -n old-k8s-version-278865
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-278865 -n old-k8s-version-278865: exit status 6 (226.432715ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0815 18:30:51.000450 67564 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-278865" does not appear in /home/jenkins/minikube-integration/19450-13013/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-278865" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (297.27s)