=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-linux-amd64 start -p old-k8s-version-108542 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0
E0725 18:39:12.056515 13059 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/functional-896905/client.crt: no such file or directory
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p old-k8s-version-108542 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0: exit status 109 (4m54.750241809s)
-- stdout --
* [old-k8s-version-108542] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=19326
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19326-5877/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19326-5877/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the kvm2 driver based on user configuration
* Starting "old-k8s-version-108542" primary control-plane node in "old-k8s-version-108542" cluster
* Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0725 18:39:08.088303 55363 out.go:291] Setting OutFile to fd 1 ...
I0725 18:39:08.088449 55363 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0725 18:39:08.088459 55363 out.go:304] Setting ErrFile to fd 2...
I0725 18:39:08.088463 55363 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0725 18:39:08.088626 55363 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19326-5877/.minikube/bin
I0725 18:39:08.089165 55363 out.go:298] Setting JSON to false
I0725 18:39:08.090144 55363 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":4892,"bootTime":1721927856,"procs":213,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0725 18:39:08.090202 55363 start.go:139] virtualization: kvm guest
I0725 18:39:08.092479 55363 out.go:177] * [old-k8s-version-108542] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0725 18:39:08.094073 55363 out.go:177] - MINIKUBE_LOCATION=19326
I0725 18:39:08.094117 55363 notify.go:220] Checking for updates...
I0725 18:39:08.096368 55363 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0725 18:39:08.097568 55363 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19326-5877/kubeconfig
I0725 18:39:08.098791 55363 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19326-5877/.minikube
I0725 18:39:08.100154 55363 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0725 18:39:08.101576 55363 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0725 18:39:08.103602 55363 config.go:182] Loaded profile config "cert-expiration-979261": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0725 18:39:08.103842 55363 config.go:182] Loaded profile config "force-systemd-flag-267077": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0725 18:39:08.103968 55363 config.go:182] Loaded profile config "kubernetes-upgrade-069209": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.31.0-beta.0
I0725 18:39:08.104098 55363 driver.go:392] Setting default libvirt URI to qemu:///system
I0725 18:39:08.140965 55363 out.go:177] * Using the kvm2 driver based on user configuration
I0725 18:39:08.142112 55363 start.go:297] selected driver: kvm2
I0725 18:39:08.142128 55363 start.go:901] validating driver "kvm2" against <nil>
I0725 18:39:08.142137 55363 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0725 18:39:08.142846 55363 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0725 18:39:08.142908 55363 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19326-5877/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0725 18:39:08.157612 55363 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0725 18:39:08.157651 55363 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0725 18:39:08.157845 55363 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0725 18:39:08.157870 55363 cni.go:84] Creating CNI manager for ""
I0725 18:39:08.157878 55363 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0725 18:39:08.157886 55363 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0725 18:39:08.157933 55363 start.go:340] cluster config:
{Name:old-k8s-version-108542 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-108542 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: S
SHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0725 18:39:08.158026 55363 iso.go:125] acquiring lock: {Name:mk0da515ffe64c6b3d23819e6e10f3a7aeecda7b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0725 18:39:08.160378 55363 out.go:177] * Starting "old-k8s-version-108542" primary control-plane node in "old-k8s-version-108542" cluster
I0725 18:39:08.161640 55363 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0725 18:39:08.161676 55363 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4
I0725 18:39:08.161683 55363 cache.go:56] Caching tarball of preloaded images
I0725 18:39:08.161767 55363 preload.go:172] Found /home/jenkins/minikube-integration/19326-5877/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0725 18:39:08.161777 55363 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0725 18:39:08.161858 55363 profile.go:143] Saving config to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/config.json ...
I0725 18:39:08.161872 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/config.json: {Name:mkfcaced8366e382cd1a238674b5b7bf3fc3fd06 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:39:08.162009 55363 start.go:360] acquireMachinesLock for old-k8s-version-108542: {Name:mk20c7a10d76951266581ba86debeac6b9496ec5 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0725 18:39:36.024884 55363 start.go:364] duration metric: took 27.86283111s to acquireMachinesLock for "old-k8s-version-108542"
I0725 18:39:36.024956 55363 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-108542 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi
g:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-108542 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions
:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0725 18:39:36.025074 55363 start.go:125] createHost starting for "" (driver="kvm2")
I0725 18:39:36.027368 55363 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...
I0725 18:39:36.027536 55363 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0725 18:39:36.027579 55363 main.go:141] libmachine: Launching plugin server for driver kvm2
I0725 18:39:36.044631 55363 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:40149
I0725 18:39:36.045014 55363 main.go:141] libmachine: () Calling .GetVersion
I0725 18:39:36.045568 55363 main.go:141] libmachine: Using API Version 1
I0725 18:39:36.045592 55363 main.go:141] libmachine: () Calling .SetConfigRaw
I0725 18:39:36.045951 55363 main.go:141] libmachine: () Calling .GetMachineName
I0725 18:39:36.046143 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetMachineName
I0725 18:39:36.048489 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:36.048758 55363 start.go:159] libmachine.API.Create for "old-k8s-version-108542" (driver="kvm2")
I0725 18:39:36.048790 55363 client.go:168] LocalClient.Create starting
I0725 18:39:36.048834 55363 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem
I0725 18:39:36.048885 55363 main.go:141] libmachine: Decoding PEM data...
I0725 18:39:36.048914 55363 main.go:141] libmachine: Parsing certificate...
I0725 18:39:36.048990 55363 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19326-5877/.minikube/certs/cert.pem
I0725 18:39:36.049020 55363 main.go:141] libmachine: Decoding PEM data...
I0725 18:39:36.049042 55363 main.go:141] libmachine: Parsing certificate...
I0725 18:39:36.049091 55363 main.go:141] libmachine: Running pre-create checks...
I0725 18:39:36.049108 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .PreCreateCheck
I0725 18:39:36.049581 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetConfigRaw
I0725 18:39:36.050010 55363 main.go:141] libmachine: Creating machine...
I0725 18:39:36.050028 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .Create
I0725 18:39:36.050169 55363 main.go:141] libmachine: (old-k8s-version-108542) Creating KVM machine...
I0725 18:39:36.051386 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found existing default KVM network
I0725 18:39:36.052990 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:36.052822 55829 network.go:206] using free private subnet 192.168.39.0/24: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000015eb0}
I0725 18:39:36.053041 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | created network xml:
I0725 18:39:36.053059 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <network>
I0725 18:39:36.053075 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <name>mk-old-k8s-version-108542</name>
I0725 18:39:36.053086 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <dns enable='no'/>
I0725 18:39:36.053096 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG |
I0725 18:39:36.053114 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <ip address='192.168.39.1' netmask='255.255.255.0'>
I0725 18:39:36.053137 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <dhcp>
I0725 18:39:36.053154 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | <range start='192.168.39.2' end='192.168.39.253'/>
I0725 18:39:36.053166 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | </dhcp>
I0725 18:39:36.053173 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | </ip>
I0725 18:39:36.053180 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG |
I0725 18:39:36.053190 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | </network>
I0725 18:39:36.053204 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG |
I0725 18:39:36.059015 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | trying to create private KVM network mk-old-k8s-version-108542 192.168.39.0/24...
I0725 18:39:36.127590 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | private KVM network mk-old-k8s-version-108542 192.168.39.0/24 created
I0725 18:39:36.127636 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting up store path in /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542 ...
I0725 18:39:36.127654 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:36.127574 55829 common.go:145] Making disk image using store path: /home/jenkins/minikube-integration/19326-5877/.minikube
I0725 18:39:36.127677 55363 main.go:141] libmachine: (old-k8s-version-108542) Building disk image from file:///home/jenkins/minikube-integration/19326-5877/.minikube/cache/iso/amd64/minikube-v1.33.1-1721690939-19319-amd64.iso
I0725 18:39:36.127744 55363 main.go:141] libmachine: (old-k8s-version-108542) Downloading /home/jenkins/minikube-integration/19326-5877/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/19326-5877/.minikube/cache/iso/amd64/minikube-v1.33.1-1721690939-19319-amd64.iso...
I0725 18:39:36.379006 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:36.378882 55829 common.go:152] Creating ssh key: /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa...
I0725 18:39:36.604909 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:36.604791 55829 common.go:158] Creating raw disk image: /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/old-k8s-version-108542.rawdisk...
I0725 18:39:36.604938 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Writing magic tar header
I0725 18:39:36.604951 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Writing SSH key tar header
I0725 18:39:36.604960 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:36.604897 55829 common.go:172] Fixing permissions on /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542 ...
I0725 18:39:36.605079 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542 (perms=drwx------)
I0725 18:39:36.605101 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins/minikube-integration/19326-5877/.minikube/machines (perms=drwxr-xr-x)
I0725 18:39:36.605113 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542
I0725 18:39:36.605128 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19326-5877/.minikube/machines
I0725 18:39:36.605148 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19326-5877/.minikube
I0725 18:39:36.605176 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins/minikube-integration/19326-5877/.minikube (perms=drwxr-xr-x)
I0725 18:39:36.605192 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins/minikube-integration/19326-5877 (perms=drwxrwxr-x)
I0725 18:39:36.605206 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0725 18:39:36.605221 55363 main.go:141] libmachine: (old-k8s-version-108542) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0725 18:39:36.605233 55363 main.go:141] libmachine: (old-k8s-version-108542) Creating domain...
I0725 18:39:36.605249 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/19326-5877
I0725 18:39:36.605265 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0725 18:39:36.605276 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home/jenkins
I0725 18:39:36.605284 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Checking permissions on dir: /home
I0725 18:39:36.605297 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Skipping /home - not owner
I0725 18:39:36.606274 55363 main.go:141] libmachine: (old-k8s-version-108542) define libvirt domain using xml:
I0725 18:39:36.606302 55363 main.go:141] libmachine: (old-k8s-version-108542) <domain type='kvm'>
I0725 18:39:36.606318 55363 main.go:141] libmachine: (old-k8s-version-108542) <name>old-k8s-version-108542</name>
I0725 18:39:36.606343 55363 main.go:141] libmachine: (old-k8s-version-108542) <memory unit='MiB'>2200</memory>
I0725 18:39:36.606356 55363 main.go:141] libmachine: (old-k8s-version-108542) <vcpu>2</vcpu>
I0725 18:39:36.606363 55363 main.go:141] libmachine: (old-k8s-version-108542) <features>
I0725 18:39:36.606374 55363 main.go:141] libmachine: (old-k8s-version-108542) <acpi/>
I0725 18:39:36.606385 55363 main.go:141] libmachine: (old-k8s-version-108542) <apic/>
I0725 18:39:36.606397 55363 main.go:141] libmachine: (old-k8s-version-108542) <pae/>
I0725 18:39:36.606410 55363 main.go:141] libmachine: (old-k8s-version-108542)
I0725 18:39:36.606437 55363 main.go:141] libmachine: (old-k8s-version-108542) </features>
I0725 18:39:36.606461 55363 main.go:141] libmachine: (old-k8s-version-108542) <cpu mode='host-passthrough'>
I0725 18:39:36.606472 55363 main.go:141] libmachine: (old-k8s-version-108542)
I0725 18:39:36.606482 55363 main.go:141] libmachine: (old-k8s-version-108542) </cpu>
I0725 18:39:36.606493 55363 main.go:141] libmachine: (old-k8s-version-108542) <os>
I0725 18:39:36.606502 55363 main.go:141] libmachine: (old-k8s-version-108542) <type>hvm</type>
I0725 18:39:36.606514 55363 main.go:141] libmachine: (old-k8s-version-108542) <boot dev='cdrom'/>
I0725 18:39:36.606523 55363 main.go:141] libmachine: (old-k8s-version-108542) <boot dev='hd'/>
I0725 18:39:36.606549 55363 main.go:141] libmachine: (old-k8s-version-108542) <bootmenu enable='no'/>
I0725 18:39:36.606569 55363 main.go:141] libmachine: (old-k8s-version-108542) </os>
I0725 18:39:36.606578 55363 main.go:141] libmachine: (old-k8s-version-108542) <devices>
I0725 18:39:36.606588 55363 main.go:141] libmachine: (old-k8s-version-108542) <disk type='file' device='cdrom'>
I0725 18:39:36.606632 55363 main.go:141] libmachine: (old-k8s-version-108542) <source file='/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/boot2docker.iso'/>
I0725 18:39:36.606649 55363 main.go:141] libmachine: (old-k8s-version-108542) <target dev='hdc' bus='scsi'/>
I0725 18:39:36.606658 55363 main.go:141] libmachine: (old-k8s-version-108542) <readonly/>
I0725 18:39:36.606673 55363 main.go:141] libmachine: (old-k8s-version-108542) </disk>
I0725 18:39:36.606685 55363 main.go:141] libmachine: (old-k8s-version-108542) <disk type='file' device='disk'>
I0725 18:39:36.606694 55363 main.go:141] libmachine: (old-k8s-version-108542) <driver name='qemu' type='raw' cache='default' io='threads' />
I0725 18:39:36.606711 55363 main.go:141] libmachine: (old-k8s-version-108542) <source file='/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/old-k8s-version-108542.rawdisk'/>
I0725 18:39:36.606718 55363 main.go:141] libmachine: (old-k8s-version-108542) <target dev='hda' bus='virtio'/>
I0725 18:39:36.606731 55363 main.go:141] libmachine: (old-k8s-version-108542) </disk>
I0725 18:39:36.606744 55363 main.go:141] libmachine: (old-k8s-version-108542) <interface type='network'>
I0725 18:39:36.606752 55363 main.go:141] libmachine: (old-k8s-version-108542) <source network='mk-old-k8s-version-108542'/>
I0725 18:39:36.606760 55363 main.go:141] libmachine: (old-k8s-version-108542) <model type='virtio'/>
I0725 18:39:36.606771 55363 main.go:141] libmachine: (old-k8s-version-108542) </interface>
I0725 18:39:36.606781 55363 main.go:141] libmachine: (old-k8s-version-108542) <interface type='network'>
I0725 18:39:36.606791 55363 main.go:141] libmachine: (old-k8s-version-108542) <source network='default'/>
I0725 18:39:36.606800 55363 main.go:141] libmachine: (old-k8s-version-108542) <model type='virtio'/>
I0725 18:39:36.606810 55363 main.go:141] libmachine: (old-k8s-version-108542) </interface>
I0725 18:39:36.606823 55363 main.go:141] libmachine: (old-k8s-version-108542) <serial type='pty'>
I0725 18:39:36.606843 55363 main.go:141] libmachine: (old-k8s-version-108542) <target port='0'/>
I0725 18:39:36.606853 55363 main.go:141] libmachine: (old-k8s-version-108542) </serial>
I0725 18:39:36.606862 55363 main.go:141] libmachine: (old-k8s-version-108542) <console type='pty'>
I0725 18:39:36.606872 55363 main.go:141] libmachine: (old-k8s-version-108542) <target type='serial' port='0'/>
I0725 18:39:36.606883 55363 main.go:141] libmachine: (old-k8s-version-108542) </console>
I0725 18:39:36.606894 55363 main.go:141] libmachine: (old-k8s-version-108542) <rng model='virtio'>
I0725 18:39:36.606904 55363 main.go:141] libmachine: (old-k8s-version-108542) <backend model='random'>/dev/random</backend>
I0725 18:39:36.606918 55363 main.go:141] libmachine: (old-k8s-version-108542) </rng>
I0725 18:39:36.606929 55363 main.go:141] libmachine: (old-k8s-version-108542)
I0725 18:39:36.606939 55363 main.go:141] libmachine: (old-k8s-version-108542)
I0725 18:39:36.606950 55363 main.go:141] libmachine: (old-k8s-version-108542) </devices>
I0725 18:39:36.606957 55363 main.go:141] libmachine: (old-k8s-version-108542) </domain>
I0725 18:39:36.606970 55363 main.go:141] libmachine: (old-k8s-version-108542)
I0725 18:39:36.611303 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:81:5d:6e in network default
I0725 18:39:36.611953 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:36.611972 55363 main.go:141] libmachine: (old-k8s-version-108542) Ensuring networks are active...
I0725 18:39:36.612727 55363 main.go:141] libmachine: (old-k8s-version-108542) Ensuring network default is active
I0725 18:39:36.613077 55363 main.go:141] libmachine: (old-k8s-version-108542) Ensuring network mk-old-k8s-version-108542 is active
I0725 18:39:36.613909 55363 main.go:141] libmachine: (old-k8s-version-108542) Getting domain xml...
I0725 18:39:36.614661 55363 main.go:141] libmachine: (old-k8s-version-108542) Creating domain...
I0725 18:39:38.136754 55363 main.go:141] libmachine: (old-k8s-version-108542) Waiting to get IP...
I0725 18:39:38.137784 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:38.138366 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:38.138392 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:38.138276 55829 retry.go:31] will retry after 255.892486ms: waiting for machine to come up
I0725 18:39:38.395922 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:38.396634 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:38.396664 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:38.396589 55829 retry.go:31] will retry after 259.432873ms: waiting for machine to come up
I0725 18:39:38.658178 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:38.658726 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:38.658756 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:38.658679 55829 retry.go:31] will retry after 470.348484ms: waiting for machine to come up
I0725 18:39:39.130233 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:39.130642 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:39.130665 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:39.130612 55829 retry.go:31] will retry after 391.408639ms: waiting for machine to come up
I0725 18:39:39.523261 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:39.523827 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:39.523853 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:39.523794 55829 retry.go:31] will retry after 698.249711ms: waiting for machine to come up
I0725 18:39:40.223635 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:40.224236 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:40.224265 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:40.224184 55829 retry.go:31] will retry after 648.316246ms: waiting for machine to come up
I0725 18:39:40.873841 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:40.874296 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:40.874337 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:40.874248 55829 retry.go:31] will retry after 1.175111915s: waiting for machine to come up
I0725 18:39:42.051218 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:42.051732 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:42.051761 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:42.051697 55829 retry.go:31] will retry after 1.22886765s: waiting for machine to come up
I0725 18:39:43.282583 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:43.283057 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:43.283089 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:43.283000 55829 retry.go:31] will retry after 1.203690762s: waiting for machine to come up
I0725 18:39:44.488534 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:44.488976 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:44.489011 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:44.488944 55829 retry.go:31] will retry after 1.88856032s: waiting for machine to come up
I0725 18:39:46.379318 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:46.379805 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:46.379835 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:46.379747 55829 retry.go:31] will retry after 1.867968999s: waiting for machine to come up
I0725 18:39:48.249726 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:48.250358 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:48.250387 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:48.250287 55829 retry.go:31] will retry after 2.336950458s: waiting for machine to come up
I0725 18:39:50.589457 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:50.590022 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:50.590053 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:50.589945 55829 retry.go:31] will retry after 3.235588362s: waiting for machine to come up
I0725 18:39:53.827361 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:53.827864 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find current IP address of domain old-k8s-version-108542 in network mk-old-k8s-version-108542
I0725 18:39:53.827887 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | I0725 18:39:53.827809 55829 retry.go:31] will retry after 3.500004092s: waiting for machine to come up
I0725 18:39:57.329555 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.330152 55363 main.go:141] libmachine: (old-k8s-version-108542) Found IP for machine: 192.168.39.29
I0725 18:39:57.330175 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has current primary IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.330183 55363 main.go:141] libmachine: (old-k8s-version-108542) Reserving static IP address...
I0725 18:39:57.330607 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | unable to find host DHCP lease matching {name: "old-k8s-version-108542", mac: "52:54:00:19:68:38", ip: "192.168.39.29"} in network mk-old-k8s-version-108542
I0725 18:39:57.408265 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Getting to WaitForSSH function...
I0725 18:39:57.408311 55363 main.go:141] libmachine: (old-k8s-version-108542) Reserved static IP address: 192.168.39.29
I0725 18:39:57.408373 55363 main.go:141] libmachine: (old-k8s-version-108542) Waiting for SSH to be available...
I0725 18:39:57.411276 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.411733 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:minikube Clientid:01:52:54:00:19:68:38}
I0725 18:39:57.411757 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.411955 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Using SSH client type: external
I0725 18:39:57.411983 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Using SSH private key: /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa (-rw-------)
I0725 18:39:57.412022 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.39.29 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa -p 22] /usr/bin/ssh <nil>}
I0725 18:39:57.412033 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | About to run SSH command:
I0725 18:39:57.412075 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | exit 0
I0725 18:39:57.544664 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | SSH cmd err, output: <nil>:
I0725 18:39:57.544944 55363 main.go:141] libmachine: (old-k8s-version-108542) KVM machine creation complete!
I0725 18:39:57.545303 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetConfigRaw
I0725 18:39:57.545986 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:57.546201 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:57.546394 55363 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...
I0725 18:39:57.546412 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetState
I0725 18:39:57.547916 55363 main.go:141] libmachine: Detecting operating system of created instance...
I0725 18:39:57.547935 55363 main.go:141] libmachine: Waiting for SSH to be available...
I0725 18:39:57.547944 55363 main.go:141] libmachine: Getting to WaitForSSH function...
I0725 18:39:57.547954 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:57.550582 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.551037 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:57.551079 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.551230 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:57.551402 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.551534 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.551698 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:57.551894 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:57.552076 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:57.552088 55363 main.go:141] libmachine: About to run SSH command:
exit 0
I0725 18:39:57.667498 55363 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0725 18:39:57.667531 55363 main.go:141] libmachine: Detecting the provisioner...
I0725 18:39:57.667541 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:57.670600 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.671047 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:57.671080 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.671197 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:57.671394 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.671579 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.671763 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:57.671923 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:57.672243 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:57.672259 55363 main.go:141] libmachine: About to run SSH command:
cat /etc/os-release
I0725 18:39:57.784812 55363 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot
VERSION=2023.02.9-dirty
ID=buildroot
VERSION_ID=2023.02.9
PRETTY_NAME="Buildroot 2023.02.9"
I0725 18:39:57.784871 55363 main.go:141] libmachine: found compatible host: buildroot
I0725 18:39:57.784879 55363 main.go:141] libmachine: Provisioning with buildroot...
I0725 18:39:57.784890 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetMachineName
I0725 18:39:57.785189 55363 buildroot.go:166] provisioning hostname "old-k8s-version-108542"
I0725 18:39:57.785222 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetMachineName
I0725 18:39:57.785436 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:57.788306 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.788747 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:57.788788 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.788974 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:57.789142 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.789331 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.789475 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:57.789673 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:57.789898 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:57.789916 55363 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-108542 && echo "old-k8s-version-108542" | sudo tee /etc/hostname
I0725 18:39:57.917676 55363 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-108542
I0725 18:39:57.917713 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:57.920786 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.921301 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:57.921334 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:57.921518 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:57.921725 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.921942 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:57.922087 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:57.922291 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:57.922498 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:57.922522 55363 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-108542' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-108542/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-108542' | sudo tee -a /etc/hosts;
fi
fi
I0725 18:39:58.053079 55363 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0725 18:39:58.053111 55363 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19326-5877/.minikube CaCertPath:/home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19326-5877/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19326-5877/.minikube}
I0725 18:39:58.053172 55363 buildroot.go:174] setting up certificates
I0725 18:39:58.053182 55363 provision.go:84] configureAuth start
I0725 18:39:58.053204 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetMachineName
I0725 18:39:58.053513 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetIP
I0725 18:39:58.056481 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.056860 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.056891 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.057079 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:58.059354 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.059764 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.059789 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.059991 55363 provision.go:143] copyHostCerts
I0725 18:39:58.060076 55363 exec_runner.go:144] found /home/jenkins/minikube-integration/19326-5877/.minikube/ca.pem, removing ...
I0725 18:39:58.060096 55363 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19326-5877/.minikube/ca.pem
I0725 18:39:58.060169 55363 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19326-5877/.minikube/ca.pem (1078 bytes)
I0725 18:39:58.060345 55363 exec_runner.go:144] found /home/jenkins/minikube-integration/19326-5877/.minikube/cert.pem, removing ...
I0725 18:39:58.060360 55363 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19326-5877/.minikube/cert.pem
I0725 18:39:58.060402 55363 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19326-5877/.minikube/cert.pem (1123 bytes)
I0725 18:39:58.060506 55363 exec_runner.go:144] found /home/jenkins/minikube-integration/19326-5877/.minikube/key.pem, removing ...
I0725 18:39:58.060517 55363 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19326-5877/.minikube/key.pem
I0725 18:39:58.060546 55363 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19326-5877/.minikube/key.pem (1679 bytes)
I0725 18:39:58.060618 55363 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19326-5877/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-108542 san=[127.0.0.1 192.168.39.29 localhost minikube old-k8s-version-108542]
I0725 18:39:58.385792 55363 provision.go:177] copyRemoteCerts
I0725 18:39:58.385865 55363 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0725 18:39:58.385913 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:58.389196 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.389616 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.389646 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.389890 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:58.390102 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.390308 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:58.390457 55363 sshutil.go:53] new ssh client: &{IP:192.168.39.29 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa Username:docker}
I0725 18:39:58.478436 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0725 18:39:58.501251 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0725 18:39:58.527373 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0725 18:39:58.549842 55363 provision.go:87] duration metric: took 496.643249ms to configureAuth
I0725 18:39:58.549872 55363 buildroot.go:189] setting minikube options for container-runtime
I0725 18:39:58.550076 55363 config.go:182] Loaded profile config "old-k8s-version-108542": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0725 18:39:58.550159 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:58.552643 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.553003 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.553034 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.553164 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:58.553368 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.553557 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.553700 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:58.553938 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:58.554160 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:58.554177 55363 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0725 18:39:58.825636 55363 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0725 18:39:58.825665 55363 main.go:141] libmachine: Checking connection to Docker...
I0725 18:39:58.825676 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetURL
I0725 18:39:58.826956 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | Using libvirt version 6000000
I0725 18:39:58.829526 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.829895 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.829917 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.830108 55363 main.go:141] libmachine: Docker is up and running!
I0725 18:39:58.830124 55363 main.go:141] libmachine: Reticulating splines...
I0725 18:39:58.830131 55363 client.go:171] duration metric: took 22.781331722s to LocalClient.Create
I0725 18:39:58.830158 55363 start.go:167] duration metric: took 22.781400806s to libmachine.API.Create "old-k8s-version-108542"
I0725 18:39:58.830171 55363 start.go:293] postStartSetup for "old-k8s-version-108542" (driver="kvm2")
I0725 18:39:58.830205 55363 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0725 18:39:58.830227 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:58.830470 55363 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0725 18:39:58.830495 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:58.832941 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.833399 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.833426 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.833564 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:58.833719 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.833856 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:58.833990 55363 sshutil.go:53] new ssh client: &{IP:192.168.39.29 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa Username:docker}
I0725 18:39:58.918647 55363 ssh_runner.go:195] Run: cat /etc/os-release
I0725 18:39:58.922535 55363 info.go:137] Remote host: Buildroot 2023.02.9
I0725 18:39:58.922561 55363 filesync.go:126] Scanning /home/jenkins/minikube-integration/19326-5877/.minikube/addons for local assets ...
I0725 18:39:58.922626 55363 filesync.go:126] Scanning /home/jenkins/minikube-integration/19326-5877/.minikube/files for local assets ...
I0725 18:39:58.922709 55363 filesync.go:149] local asset: /home/jenkins/minikube-integration/19326-5877/.minikube/files/etc/ssl/certs/130592.pem -> 130592.pem in /etc/ssl/certs
I0725 18:39:58.922795 55363 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0725 18:39:58.931764 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/files/etc/ssl/certs/130592.pem --> /etc/ssl/certs/130592.pem (1708 bytes)
I0725 18:39:58.954933 55363 start.go:296] duration metric: took 124.733843ms for postStartSetup
I0725 18:39:58.954985 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetConfigRaw
I0725 18:39:58.955577 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetIP
I0725 18:39:58.958702 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.959459 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.959496 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.959717 55363 profile.go:143] Saving config to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/config.json ...
I0725 18:39:58.959955 55363 start.go:128] duration metric: took 22.93486958s to createHost
I0725 18:39:58.959982 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:58.962374 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.962692 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:58.962719 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:58.962843 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:58.963023 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.963240 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:58.963443 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:58.963592 55363 main.go:141] libmachine: Using SSH client type: native
I0725 18:39:58.963772 55363 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.29 22 <nil> <nil>}
I0725 18:39:58.963858 55363 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0725 18:39:59.076848 55363 main.go:141] libmachine: SSH cmd err, output: <nil>: 1721932799.042325621
I0725 18:39:59.076869 55363 fix.go:216] guest clock: 1721932799.042325621
I0725 18:39:59.076878 55363 fix.go:229] Guest: 2024-07-25 18:39:59.042325621 +0000 UTC Remote: 2024-07-25 18:39:58.959970358 +0000 UTC m=+50.903762414 (delta=82.355263ms)
I0725 18:39:59.076925 55363 fix.go:200] guest clock delta is within tolerance: 82.355263ms
I0725 18:39:59.076933 55363 start.go:83] releasing machines lock for "old-k8s-version-108542", held for 23.052020473s
I0725 18:39:59.076967 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:59.077243 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetIP
I0725 18:39:59.080294 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.080660 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:59.080689 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.080923 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:59.081563 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:59.081738 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .DriverName
I0725 18:39:59.081816 55363 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0725 18:39:59.081870 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:59.082011 55363 ssh_runner.go:195] Run: cat /version.json
I0725 18:39:59.082040 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHHostname
I0725 18:39:59.085042 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.085208 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.085413 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:59.085442 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.085604 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:59.085615 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:39:59.085651 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:39:59.085775 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:59.085838 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHPort
I0725 18:39:59.085947 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:59.086033 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHKeyPath
I0725 18:39:59.086107 55363 sshutil.go:53] new ssh client: &{IP:192.168.39.29 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa Username:docker}
I0725 18:39:59.086469 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetSSHUsername
I0725 18:39:59.086690 55363 sshutil.go:53] new ssh client: &{IP:192.168.39.29 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19326-5877/.minikube/machines/old-k8s-version-108542/id_rsa Username:docker}
I0725 18:39:59.209375 55363 ssh_runner.go:195] Run: systemctl --version
I0725 18:39:59.215649 55363 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0725 18:39:59.376812 55363 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0725 18:39:59.382914 55363 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0725 18:39:59.382994 55363 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0725 18:39:59.399576 55363 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0725 18:39:59.399602 55363 start.go:495] detecting cgroup driver to use...
I0725 18:39:59.399665 55363 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0725 18:39:59.417234 55363 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0725 18:39:59.430697 55363 docker.go:217] disabling cri-docker service (if available) ...
I0725 18:39:59.430764 55363 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0725 18:39:59.446466 55363 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0725 18:39:59.460997 55363 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0725 18:39:59.585882 55363 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0725 18:39:59.730360 55363 docker.go:233] disabling docker service ...
I0725 18:39:59.730417 55363 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0725 18:39:59.748258 55363 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0725 18:39:59.761130 55363 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0725 18:39:59.905267 55363 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0725 18:40:00.024831 55363 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0725 18:40:00.039802 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0725 18:40:00.057521 55363 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0725 18:40:00.057574 55363 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0725 18:40:00.066917 55363 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0725 18:40:00.066992 55363 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0725 18:40:00.076664 55363 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0725 18:40:00.086490 55363 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0725 18:40:00.095845 55363 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0725 18:40:00.105984 55363 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0725 18:40:00.114771 55363 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 255
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I0725 18:40:00.114833 55363 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I0725 18:40:00.127039 55363 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0725 18:40:00.136671 55363 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0725 18:40:00.266606 55363 ssh_runner.go:195] Run: sudo systemctl restart crio
I0725 18:40:00.413007 55363 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0725 18:40:00.413084 55363 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0725 18:40:00.417628 55363 start.go:563] Will wait 60s for crictl version
I0725 18:40:00.417694 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:00.421134 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0725 18:40:00.460209 55363 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0725 18:40:00.460295 55363 ssh_runner.go:195] Run: crio --version
I0725 18:40:00.487130 55363 ssh_runner.go:195] Run: crio --version
I0725 18:40:00.519004 55363 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.29.1 ...
I0725 18:40:00.520234 55363 main.go:141] libmachine: (old-k8s-version-108542) Calling .GetIP
I0725 18:40:00.523718 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:40:00.524159 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:19:68:38", ip: ""} in network mk-old-k8s-version-108542: {Iface:virbr3 ExpiryTime:2024-07-25 19:39:50 +0000 UTC Type:0 Mac:52:54:00:19:68:38 Iaid: IPaddr:192.168.39.29 Prefix:24 Hostname:old-k8s-version-108542 Clientid:01:52:54:00:19:68:38}
I0725 18:40:00.524195 55363 main.go:141] libmachine: (old-k8s-version-108542) DBG | domain old-k8s-version-108542 has defined IP address 192.168.39.29 and MAC address 52:54:00:19:68:38 in network mk-old-k8s-version-108542
I0725 18:40:00.524432 55363 ssh_runner.go:195] Run: grep 192.168.39.1 host.minikube.internal$ /etc/hosts
I0725 18:40:00.529740 55363 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.39.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0725 18:40:00.545736 55363 kubeadm.go:883] updating cluster {Name:old-k8s-version-108542 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersio
n:v1.20.0 ClusterName:old-k8s-version-108542 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.29 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0725 18:40:00.545874 55363 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0725 18:40:00.545935 55363 ssh_runner.go:195] Run: sudo crictl images --output json
I0725 18:40:00.584454 55363 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0725 18:40:00.584531 55363 ssh_runner.go:195] Run: which lz4
I0725 18:40:00.588881 55363 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0725 18:40:00.592738 55363 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0725 18:40:00.592795 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-amd64.tar.lz4 --> /preloaded.tar.lz4 (473237281 bytes)
I0725 18:40:02.105945 55363 crio.go:462] duration metric: took 1.517092405s to copy over tarball
I0725 18:40:02.106041 55363 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0725 18:40:04.747559 55363 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (2.641480879s)
I0725 18:40:04.747599 55363 crio.go:469] duration metric: took 2.641617846s to extract the tarball
I0725 18:40:04.747610 55363 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0725 18:40:04.789962 55363 ssh_runner.go:195] Run: sudo crictl images --output json
I0725 18:40:04.835104 55363 crio.go:510] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.20.0". assuming images are not preloaded.
I0725 18:40:04.835134 55363 cache_images.go:88] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.20.0 registry.k8s.io/kube-controller-manager:v1.20.0 registry.k8s.io/kube-scheduler:v1.20.0 registry.k8s.io/kube-proxy:v1.20.0 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.13-0 registry.k8s.io/coredns:1.7.0 gcr.io/k8s-minikube/storage-provisioner:v5]
I0725 18:40:04.835204 55363 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0725 18:40:04.835261 55363 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.20.0
I0725 18:40:04.835269 55363 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.13-0
I0725 18:40:04.835283 55363 image.go:134] retrieving image: registry.k8s.io/coredns:1.7.0
I0725 18:40:04.835244 55363 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.20.0
I0725 18:40:04.835325 55363 image.go:134] retrieving image: registry.k8s.io/pause:3.2
I0725 18:40:04.835244 55363 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.20.0
I0725 18:40:04.835561 55363 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.20.0
I0725 18:40:04.836832 55363 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2
I0725 18:40:04.836851 55363 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.20.0
I0725 18:40:04.836858 55363 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.20.0
I0725 18:40:04.836831 55363 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0725 18:40:04.836877 55363 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.20.0
I0725 18:40:04.836832 55363 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.20.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.20.0
I0725 18:40:04.836835 55363 image.go:177] daemon lookup for registry.k8s.io/coredns:1.7.0: Error response from daemon: No such image: registry.k8s.io/coredns:1.7.0
I0725 18:40:04.837185 55363 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.13-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.13-0
I0725 18:40:05.063753 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.13-0
I0725 18:40:05.074051 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/pause:3.2
I0725 18:40:05.092907 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.20.0
I0725 18:40:05.093187 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.20.0
I0725 18:40:05.095225 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/coredns:1.7.0
I0725 18:40:05.106782 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.20.0
I0725 18:40:05.146697 55363 cache_images.go:116] "registry.k8s.io/etcd:3.4.13-0" needs transfer: "registry.k8s.io/etcd:3.4.13-0" does not exist at hash "0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934" in container runtime
I0725 18:40:05.146761 55363 cri.go:218] Removing image: registry.k8s.io/etcd:3.4.13-0
I0725 18:40:05.146829 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.159970 55363 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime
I0725 18:40:05.160009 55363 cri.go:218] Removing image: registry.k8s.io/pause:3.2
I0725 18:40:05.160052 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.199196 55363 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.20.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.20.0" does not exist at hash "ca9843d3b545457f24b012d6d579ba85f132f2406aa171ad84d53caa55e5de99" in container runtime
I0725 18:40:05.199241 55363 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.20.0
I0725 18:40:05.199291 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.200833 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.20.0
I0725 18:40:05.250337 55363 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.20.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.20.0" does not exist at hash "3138b6e3d471224fd516f758f3b53309219bcb6824e07686b3cd60d78012c899" in container runtime
I0725 18:40:05.250385 55363 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.20.0
I0725 18:40:05.250435 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.256798 55363 cache_images.go:116] "registry.k8s.io/coredns:1.7.0" needs transfer: "registry.k8s.io/coredns:1.7.0" does not exist at hash "bfe3a36ebd2528b454be6aebece806db5b40407b833e2af9617bf39afaff8c16" in container runtime
I0725 18:40:05.256840 55363 cri.go:218] Removing image: registry.k8s.io/coredns:1.7.0
I0725 18:40:05.256859 55363 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.20.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.20.0" does not exist at hash "b9fa1895dcaa6d3dd241d6d9340e939ca30fc0946464ec9f205a8cbe738a8080" in container runtime
I0725 18:40:05.256880 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.256888 55363 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.20.0
I0725 18:40:05.256912 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.4.13-0
I0725 18:40:05.256931 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.2
I0725 18:40:05.256936 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.257017 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.20.0
I0725 18:40:05.274907 55363 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.20.0" needs transfer: "registry.k8s.io/kube-proxy:v1.20.0" does not exist at hash "10cc881966cfd9287656c2fce1f144625602653d1e8b011487a7a71feb100bdc" in container runtime
I0725 18:40:05.274948 55363 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.20.0
I0725 18:40:05.274944 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.20.0
I0725 18:40:05.275006 55363 ssh_runner.go:195] Run: which crictl
I0725 18:40:05.275947 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns:1.7.0
I0725 18:40:05.365454 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.20.0
I0725 18:40:05.365502 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0
I0725 18:40:05.365541 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2
I0725 18:40:05.365593 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.20.0
I0725 18:40:05.365601 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.20.0
I0725 18:40:05.365652 55363 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.20.0
I0725 18:40:05.375451 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.7.0
I0725 18:40:05.410490 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.20.0
I0725 18:40:05.410755 55363 cache_images.go:289] Loading image from: /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.20.0
I0725 18:40:05.679221 55363 ssh_runner.go:195] Run: sudo podman image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0725 18:40:05.824392 55363 cache_images.go:92] duration metric: took 989.23997ms to LoadCachedImages
W0725 18:40:05.824487 55363 out.go:239] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/19326-5877/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.13-0: no such file or directory
I0725 18:40:05.824505 55363 kubeadm.go:934] updating node { 192.168.39.29 8443 v1.20.0 crio true true} ...
I0725 18:40:05.824653 55363 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --hostname-override=old-k8s-version-108542 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.39.29
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-108542 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0725 18:40:05.824735 55363 ssh_runner.go:195] Run: crio config
I0725 18:40:05.872925 55363 cni.go:84] Creating CNI manager for ""
I0725 18:40:05.872942 55363 cni.go:146] "kvm2" driver + "crio" runtime found, recommending bridge
I0725 18:40:05.872950 55363 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0725 18:40:05.872967 55363 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.29 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-108542 NodeName:old-k8s-version-108542 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.29"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.39.29 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt Stati
cPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0725 18:40:05.873082 55363 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.39.29
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-108542"
kubeletExtraArgs:
node-ip: 192.168.39.29
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.39.29"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0725 18:40:05.873156 55363 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0725 18:40:05.883172 55363 binaries.go:44] Found k8s binaries, skipping transfer
I0725 18:40:05.883242 55363 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0725 18:40:05.892444 55363 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (429 bytes)
I0725 18:40:05.910094 55363 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0725 18:40:05.927108 55363 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0725 18:40:05.944764 55363 ssh_runner.go:195] Run: grep 192.168.39.29 control-plane.minikube.internal$ /etc/hosts
I0725 18:40:05.948359 55363 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.39.29 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0725 18:40:05.960525 55363 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0725 18:40:06.091832 55363 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0725 18:40:06.108764 55363 certs.go:68] Setting up /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542 for IP: 192.168.39.29
I0725 18:40:06.108788 55363 certs.go:194] generating shared ca certs ...
I0725 18:40:06.108807 55363 certs.go:226] acquiring lock for ca certs: {Name:mkae961b8e7098592ad63fee7e911c3a838ab04a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.108952 55363 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19326-5877/.minikube/ca.key
I0725 18:40:06.109018 55363 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19326-5877/.minikube/proxy-client-ca.key
I0725 18:40:06.109030 55363 certs.go:256] generating profile certs ...
I0725 18:40:06.109096 55363 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.key
I0725 18:40:06.109114 55363 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.crt with IP's: []
I0725 18:40:06.211721 55363 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.crt ...
I0725 18:40:06.211754 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.crt: {Name:mk8328536e6d3e3be7b69becd8ce6118480d4a14 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.211946 55363 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.key ...
I0725 18:40:06.211965 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/client.key: {Name:mk8e33c79977a60da7b73fdc37309f0c31106033 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.212070 55363 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key.da8b5ed0
I0725 18:40:06.212090 55363 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt.da8b5ed0 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.39.29]
I0725 18:40:06.367802 55363 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt.da8b5ed0 ...
I0725 18:40:06.367833 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt.da8b5ed0: {Name:mk91608cd2a2de482eeb1632fee3d4305bd1201d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.368013 55363 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key.da8b5ed0 ...
I0725 18:40:06.368031 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key.da8b5ed0: {Name:mk4bb7258cc724f63f302746925003a7acfe5435 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.368122 55363 certs.go:381] copying /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt.da8b5ed0 -> /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt
I0725 18:40:06.368242 55363 certs.go:385] copying /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key.da8b5ed0 -> /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key
I0725 18:40:06.368345 55363 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.key
I0725 18:40:06.368369 55363 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.crt with IP's: []
I0725 18:40:06.502724 55363 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.crt ...
I0725 18:40:06.502756 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.crt: {Name:mk92d2a6177ff7a114ce9ed043355ebaa1c7b554 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.582324 55363 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.key ...
I0725 18:40:06.582381 55363 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.key: {Name:mk0843904be2b18411f9215c4b88ee807d70f9ab Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0725 18:40:06.582628 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/13059.pem (1338 bytes)
W0725 18:40:06.582680 55363 certs.go:480] ignoring /home/jenkins/minikube-integration/19326-5877/.minikube/certs/13059_empty.pem, impossibly tiny 0 bytes
I0725 18:40:06.582696 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca-key.pem (1679 bytes)
I0725 18:40:06.582724 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/ca.pem (1078 bytes)
I0725 18:40:06.582752 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/cert.pem (1123 bytes)
I0725 18:40:06.582776 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/certs/key.pem (1679 bytes)
I0725 18:40:06.582823 55363 certs.go:484] found cert: /home/jenkins/minikube-integration/19326-5877/.minikube/files/etc/ssl/certs/130592.pem (1708 bytes)
I0725 18:40:06.583537 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0725 18:40:06.610789 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0725 18:40:06.637514 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0725 18:40:06.666644 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0725 18:40:06.689890 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0725 18:40:06.729398 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0725 18:40:06.755382 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0725 18:40:06.780347 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/profiles/old-k8s-version-108542/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0725 18:40:06.805821 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/certs/13059.pem --> /usr/share/ca-certificates/13059.pem (1338 bytes)
I0725 18:40:06.828814 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/files/etc/ssl/certs/130592.pem --> /usr/share/ca-certificates/130592.pem (1708 bytes)
I0725 18:40:06.852649 55363 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19326-5877/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0725 18:40:06.876771 55363 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0725 18:40:06.893439 55363 ssh_runner.go:195] Run: openssl version
I0725 18:40:06.898916 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/13059.pem && ln -fs /usr/share/ca-certificates/13059.pem /etc/ssl/certs/13059.pem"
I0725 18:40:06.909715 55363 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/13059.pem
I0725 18:40:06.913779 55363 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jul 25 17:41 /usr/share/ca-certificates/13059.pem
I0725 18:40:06.913835 55363 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/13059.pem
I0725 18:40:06.919070 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/13059.pem /etc/ssl/certs/51391683.0"
I0725 18:40:06.929184 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/130592.pem && ln -fs /usr/share/ca-certificates/130592.pem /etc/ssl/certs/130592.pem"
I0725 18:40:06.939501 55363 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/130592.pem
I0725 18:40:06.943820 55363 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jul 25 17:41 /usr/share/ca-certificates/130592.pem
I0725 18:40:06.943885 55363 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/130592.pem
I0725 18:40:06.949830 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/130592.pem /etc/ssl/certs/3ec20f2e.0"
I0725 18:40:06.963133 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0725 18:40:06.974988 55363 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0725 18:40:06.983118 55363 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jul 25 17:30 /usr/share/ca-certificates/minikubeCA.pem
I0725 18:40:06.983196 55363 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0725 18:40:06.992387 55363 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0725 18:40:07.006291 55363 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0725 18:40:07.010904 55363 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0725 18:40:07.010974 55363 kubeadm.go:392] StartCluster: {Name:old-k8s-version-108542 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v
1.20.0 ClusterName:old-k8s-version-108542 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.29 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mount
Port:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0725 18:40:07.011080 55363 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0725 18:40:07.011158 55363 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0725 18:40:07.065948 55363 cri.go:89] found id: ""
I0725 18:40:07.066029 55363 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0725 18:40:07.076406 55363 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0725 18:40:07.085776 55363 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0725 18:40:07.094830 55363 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0725 18:40:07.094850 55363 kubeadm.go:157] found existing configuration files:
I0725 18:40:07.094890 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0725 18:40:07.103810 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0725 18:40:07.103880 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0725 18:40:07.113126 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0725 18:40:07.121382 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0725 18:40:07.121441 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0725 18:40:07.129987 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0725 18:40:07.138871 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0725 18:40:07.138935 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0725 18:40:07.147572 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0725 18:40:07.159201 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0725 18:40:07.159267 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0725 18:40:07.168883 55363 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0725 18:40:07.286011 55363 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0725 18:40:07.286257 55363 kubeadm.go:310] [preflight] Running pre-flight checks
I0725 18:40:07.441437 55363 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0725 18:40:07.441654 55363 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0725 18:40:07.441804 55363 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0725 18:40:07.636278 55363 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0725 18:40:07.848915 55363 out.go:204] - Generating certificates and keys ...
I0725 18:40:07.849053 55363 kubeadm.go:310] [certs] Using existing ca certificate authority
I0725 18:40:07.849155 55363 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0725 18:40:07.849254 55363 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0725 18:40:07.927608 55363 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0725 18:40:08.047052 55363 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0725 18:40:08.153679 55363 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0725 18:40:08.307176 55363 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0725 18:40:08.307492 55363 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
I0725 18:40:08.375108 55363 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0725 18:40:08.375273 55363 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
I0725 18:40:08.586311 55363 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0725 18:40:08.691328 55363 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0725 18:40:08.744404 55363 kubeadm.go:310] [certs] Generating "sa" key and public key
I0725 18:40:08.744656 55363 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0725 18:40:08.956232 55363 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0725 18:40:09.604946 55363 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0725 18:40:09.900696 55363 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0725 18:40:10.076436 55363 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0725 18:40:10.094477 55363 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0725 18:40:10.094611 55363 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0725 18:40:10.094673 55363 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0725 18:40:10.240894 55363 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0725 18:40:10.397143 55363 out.go:204] - Booting up control plane ...
I0725 18:40:10.397296 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0725 18:40:10.397398 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0725 18:40:10.397492 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0725 18:40:10.397598 55363 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0725 18:40:10.397786 55363 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0725 18:40:50.253327 55363 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0725 18:40:50.253457 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:40:50.253749 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:40:55.253214 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:40:55.253525 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:41:05.252804 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:41:05.253106 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:41:25.253381 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:41:25.253547 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:42:05.255183 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:42:05.255426 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:42:05.255441 55363 kubeadm.go:310]
I0725 18:42:05.255526 55363 kubeadm.go:310] Unfortunately, an error has occurred:
I0725 18:42:05.255562 55363 kubeadm.go:310] timed out waiting for the condition
I0725 18:42:05.255568 55363 kubeadm.go:310]
I0725 18:42:05.255628 55363 kubeadm.go:310] This error is likely caused by:
I0725 18:42:05.255683 55363 kubeadm.go:310] - The kubelet is not running
I0725 18:42:05.255824 55363 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0725 18:42:05.255837 55363 kubeadm.go:310]
I0725 18:42:05.255979 55363 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0725 18:42:05.256022 55363 kubeadm.go:310] - 'systemctl status kubelet'
I0725 18:42:05.256054 55363 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0725 18:42:05.256063 55363 kubeadm.go:310]
I0725 18:42:05.256188 55363 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0725 18:42:05.256301 55363 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0725 18:42:05.256313 55363 kubeadm.go:310]
I0725 18:42:05.256473 55363 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0725 18:42:05.256598 55363 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0725 18:42:05.256701 55363 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0725 18:42:05.256786 55363 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0725 18:42:05.256802 55363 kubeadm.go:310]
I0725 18:42:05.257329 55363 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0725 18:42:05.257430 55363 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0725 18:42:05.257484 55363 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
W0725 18:42:05.257593 55363 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost old-k8s-version-108542] and IPs [192.168.39.29 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0725 18:42:05.257636 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I0725 18:42:05.731633 55363 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0725 18:42:05.745419 55363 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0725 18:42:05.754584 55363 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0725 18:42:05.754605 55363 kubeadm.go:157] found existing configuration files:
I0725 18:42:05.754647 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0725 18:42:05.763805 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0725 18:42:05.763866 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0725 18:42:05.772913 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0725 18:42:05.782261 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0725 18:42:05.782326 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0725 18:42:05.791477 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0725 18:42:05.800294 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0725 18:42:05.800374 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0725 18:42:05.809436 55363 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0725 18:42:05.819069 55363 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0725 18:42:05.819118 55363 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0725 18:42:05.829080 55363 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"
I0725 18:42:05.903428 55363 kubeadm.go:310] [init] Using Kubernetes version: v1.20.0
I0725 18:42:05.903507 55363 kubeadm.go:310] [preflight] Running pre-flight checks
I0725 18:42:06.068883 55363 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0725 18:42:06.069013 55363 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0725 18:42:06.069150 55363 kubeadm.go:310] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0725 18:42:06.260471 55363 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0725 18:42:06.263150 55363 out.go:204] - Generating certificates and keys ...
I0725 18:42:06.263255 55363 kubeadm.go:310] [certs] Using existing ca certificate authority
I0725 18:42:06.263354 55363 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0725 18:42:06.263430 55363 kubeadm.go:310] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0725 18:42:06.263512 55363 kubeadm.go:310] [certs] Using existing front-proxy-ca certificate authority
I0725 18:42:06.263607 55363 kubeadm.go:310] [certs] Using existing front-proxy-client certificate and key on disk
I0725 18:42:06.263680 55363 kubeadm.go:310] [certs] Using existing etcd/ca certificate authority
I0725 18:42:06.263735 55363 kubeadm.go:310] [certs] Using existing etcd/server certificate and key on disk
I0725 18:42:06.263826 55363 kubeadm.go:310] [certs] Using existing etcd/peer certificate and key on disk
I0725 18:42:06.263937 55363 kubeadm.go:310] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0725 18:42:06.264053 55363 kubeadm.go:310] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0725 18:42:06.264125 55363 kubeadm.go:310] [certs] Using the existing "sa" key
I0725 18:42:06.264199 55363 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0725 18:42:06.355034 55363 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0725 18:42:06.599751 55363 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0725 18:42:06.826783 55363 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0725 18:42:07.018500 55363 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0725 18:42:07.035467 55363 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0725 18:42:07.036096 55363 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0725 18:42:07.036175 55363 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0725 18:42:07.157333 55363 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0725 18:42:07.159056 55363 out.go:204] - Booting up control plane ...
I0725 18:42:07.159151 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0725 18:42:07.165687 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0725 18:42:07.166578 55363 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0725 18:42:07.167391 55363 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0725 18:42:07.169490 55363 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0725 18:42:47.167664 55363 kubeadm.go:310] [kubelet-check] Initial timeout of 40s passed.
I0725 18:42:47.168095 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:42:47.168265 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:42:52.168336 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:42:52.168582 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:43:02.168504 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:43:02.168803 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:43:22.169267 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:43:22.169514 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:44:02.171666 55363 kubeadm.go:310] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0725 18:44:02.171953 55363 kubeadm.go:310] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0725 18:44:02.171968 55363 kubeadm.go:310]
I0725 18:44:02.172038 55363 kubeadm.go:310] Unfortunately, an error has occurred:
I0725 18:44:02.172099 55363 kubeadm.go:310] timed out waiting for the condition
I0725 18:44:02.172109 55363 kubeadm.go:310]
I0725 18:44:02.172146 55363 kubeadm.go:310] This error is likely caused by:
I0725 18:44:02.172186 55363 kubeadm.go:310] - The kubelet is not running
I0725 18:44:02.172371 55363 kubeadm.go:310] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0725 18:44:02.172393 55363 kubeadm.go:310]
I0725 18:44:02.172513 55363 kubeadm.go:310] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0725 18:44:02.172545 55363 kubeadm.go:310] - 'systemctl status kubelet'
I0725 18:44:02.172575 55363 kubeadm.go:310] - 'journalctl -xeu kubelet'
I0725 18:44:02.172582 55363 kubeadm.go:310]
I0725 18:44:02.172717 55363 kubeadm.go:310] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0725 18:44:02.172855 55363 kubeadm.go:310] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0725 18:44:02.172875 55363 kubeadm.go:310]
I0725 18:44:02.173029 55363 kubeadm.go:310] Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
I0725 18:44:02.173140 55363 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I0725 18:44:02.173271 55363 kubeadm.go:310] Once you have found the failing container, you can inspect its logs with:
I0725 18:44:02.173372 55363 kubeadm.go:310] - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
I0725 18:44:02.173382 55363 kubeadm.go:310]
I0725 18:44:02.174025 55363 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0725 18:44:02.174117 55363 kubeadm.go:310] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0725 18:44:02.174260 55363 kubeadm.go:310] To see the stack trace of this error execute with --v=5 or higher
I0725 18:44:02.174305 55363 kubeadm.go:394] duration metric: took 3m55.163334882s to StartCluster
I0725 18:44:02.174352 55363 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0725 18:44:02.174416 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0725 18:44:02.216866 55363 cri.go:89] found id: ""
I0725 18:44:02.216894 55363 logs.go:276] 0 containers: []
W0725 18:44:02.216906 55363 logs.go:278] No container was found matching "kube-apiserver"
I0725 18:44:02.216913 55363 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0725 18:44:02.216981 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0725 18:44:02.253766 55363 cri.go:89] found id: ""
I0725 18:44:02.253794 55363 logs.go:276] 0 containers: []
W0725 18:44:02.253804 55363 logs.go:278] No container was found matching "etcd"
I0725 18:44:02.253812 55363 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0725 18:44:02.253878 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0725 18:44:02.289600 55363 cri.go:89] found id: ""
I0725 18:44:02.289630 55363 logs.go:276] 0 containers: []
W0725 18:44:02.289641 55363 logs.go:278] No container was found matching "coredns"
I0725 18:44:02.289648 55363 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0725 18:44:02.289702 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0725 18:44:02.323435 55363 cri.go:89] found id: ""
I0725 18:44:02.323464 55363 logs.go:276] 0 containers: []
W0725 18:44:02.323475 55363 logs.go:278] No container was found matching "kube-scheduler"
I0725 18:44:02.323481 55363 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0725 18:44:02.323542 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0725 18:44:02.357627 55363 cri.go:89] found id: ""
I0725 18:44:02.357694 55363 logs.go:276] 0 containers: []
W0725 18:44:02.357712 55363 logs.go:278] No container was found matching "kube-proxy"
I0725 18:44:02.357720 55363 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0725 18:44:02.357777 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0725 18:44:02.401741 55363 cri.go:89] found id: ""
I0725 18:44:02.401775 55363 logs.go:276] 0 containers: []
W0725 18:44:02.401787 55363 logs.go:278] No container was found matching "kube-controller-manager"
I0725 18:44:02.401794 55363 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0725 18:44:02.401857 55363 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0725 18:44:02.443438 55363 cri.go:89] found id: ""
I0725 18:44:02.443471 55363 logs.go:276] 0 containers: []
W0725 18:44:02.443483 55363 logs.go:278] No container was found matching "kindnet"
I0725 18:44:02.443493 55363 logs.go:123] Gathering logs for CRI-O ...
I0725 18:44:02.443507 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0725 18:44:02.557700 55363 logs.go:123] Gathering logs for container status ...
I0725 18:44:02.557732 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0725 18:44:02.598788 55363 logs.go:123] Gathering logs for kubelet ...
I0725 18:44:02.598817 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0725 18:44:02.657474 55363 logs.go:123] Gathering logs for dmesg ...
I0725 18:44:02.657508 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0725 18:44:02.672673 55363 logs.go:123] Gathering logs for describe nodes ...
I0725 18:44:02.672709 55363 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0725 18:44:02.792744 55363 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
W0725 18:44:02.792808 55363 out.go:364] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0725 18:44:02.792848 55363 out.go:239] *
*
W0725 18:44:02.792964 55363 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0725 18:44:02.792988 55363 out.go:239] *
*
W0725 18:44:02.793857 55363 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0725 18:44:02.796582 55363 out.go:177]
W0725 18:44:02.797610 55363 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.20.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.20.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0725 18:44:02.797674 55363 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0725 18:44:02.797706 55363 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0725 18:44:02.799566 55363 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-linux-amd64 start -p old-k8s-version-108542 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=kvm2 --container-runtime=crio --kubernetes-version=v1.20.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-108542 -n old-k8s-version-108542
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p old-k8s-version-108542 -n old-k8s-version-108542: exit status 6 (234.727085ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0725 18:44:03.070712 58537 status.go:417] kubeconfig endpoint: get endpoint: "old-k8s-version-108542" does not appear in /home/jenkins/minikube-integration/19326-5877/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-108542" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (295.04s)