=== RUN TestAddons/Setup
addons_test.go:108: (dbg) Run: out/minikube-linux-amd64 start -p addons-606715 --wait=true --memory=4096 --alsologtostderr --addons=registry --addons=registry-creds --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=nvidia-device-plugin --addons=yakd --addons=volcano --addons=amd-gpu-device-plugin --driver=docker --container-runtime=crio --addons=ingress --addons=ingress-dns --addons=storage-provisioner-rancher
addons_test.go:108: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p addons-606715 --wait=true --memory=4096 --alsologtostderr --addons=registry --addons=registry-creds --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=nvidia-device-plugin --addons=yakd --addons=volcano --addons=amd-gpu-device-plugin --driver=docker --container-runtime=crio --addons=ingress --addons=ingress-dns --addons=storage-provisioner-rancher: exit status 80 (8m36.322821214s)
-- stdout --
* [addons-606715] minikube v1.37.0 on Ubuntu 22.04 (kvm/amd64)
- MINIKUBE_LOCATION=21699
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/21699-407371/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/21699-407371/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker driver with root privileges
* Starting "addons-606715" primary control-plane node in "addons-606715" cluster
* Pulling base image v0.0.48-1759382731-21643 ...
* Preparing Kubernetes v1.34.1 on CRI-O 1.34.1 ...
-- /stdout --
** stderr **
I1006 10:52:35.227185 412505 out.go:360] Setting OutFile to fd 1 ...
I1006 10:52:35.227439 412505 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 10:52:35.227448 412505 out.go:374] Setting ErrFile to fd 2...
I1006 10:52:35.227452 412505 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 10:52:35.227650 412505 root.go:338] Updating PATH: /home/jenkins/minikube-integration/21699-407371/.minikube/bin
I1006 10:52:35.228158 412505 out.go:368] Setting JSON to false
I1006 10:52:35.229064 412505 start.go:130] hostinfo: {"hostname":"ubuntu-20-agent-10","uptime":5691,"bootTime":1759742264,"procs":290,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"22.04","kernelVersion":"6.8.0-1041-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I1006 10:52:35.229161 412505 start.go:140] virtualization: kvm guest
I1006 10:52:35.230978 412505 out.go:179] * [addons-606715] minikube v1.37.0 on Ubuntu 22.04 (kvm/amd64)
I1006 10:52:35.232233 412505 out.go:179] - MINIKUBE_LOCATION=21699
I1006 10:52:35.232257 412505 notify.go:220] Checking for updates...
I1006 10:52:35.234320 412505 out.go:179] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1006 10:52:35.235389 412505 out.go:179] - KUBECONFIG=/home/jenkins/minikube-integration/21699-407371/kubeconfig
I1006 10:52:35.236458 412505 out.go:179] - MINIKUBE_HOME=/home/jenkins/minikube-integration/21699-407371/.minikube
I1006 10:52:35.237391 412505 out.go:179] - MINIKUBE_BIN=out/minikube-linux-amd64
I1006 10:52:35.238306 412505 out.go:179] - MINIKUBE_FORCE_SYSTEMD=
I1006 10:52:35.239480 412505 driver.go:421] Setting default libvirt URI to qemu:///system
I1006 10:52:35.263713 412505 docker.go:123] docker version: linux-28.5.0:Docker Engine - Community
I1006 10:52:35.263825 412505 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 10:52:35.320006 412505 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:24 OomKillDisable:false NGoroutines:46 SystemTime:2025-10-06 10:52:35.30916447 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x8
6_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652162560 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-10 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[m
ap[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.40] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 10:52:35.320151 412505 docker.go:318] overlay module found
I1006 10:52:35.321893 412505 out.go:179] * Using the docker driver based on user configuration
I1006 10:52:35.322919 412505 start.go:304] selected driver: docker
I1006 10:52:35.322936 412505 start.go:924] validating driver "docker" against <nil>
I1006 10:52:35.322952 412505 start.go:935] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1006 10:52:35.323730 412505 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 10:52:35.377198 412505 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:24 OomKillDisable:false NGoroutines:46 SystemTime:2025-10-06 10:52:35.366026919 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x
86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652162560 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-10 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[
map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.40] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 10:52:35.377411 412505 start_flags.go:327] no existing cluster config was found, will generate one from the flags
I1006 10:52:35.377625 412505 start_flags.go:992] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I1006 10:52:35.379265 412505 out.go:179] * Using Docker driver with root privileges
I1006 10:52:35.380329 412505 cni.go:84] Creating CNI manager for ""
I1006 10:52:35.380401 412505 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I1006 10:52:35.380419 412505 start_flags.go:336] Found "CNI" CNI - setting NetworkPlugin=cni
I1006 10:52:35.380507 412505 start.go:348] cluster config:
{Name:addons-606715 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d Memory:4096 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.1 ClusterName:addons-606715 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime
:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.34.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:
AutoPauseInterval:1m0s}
I1006 10:52:35.381622 412505 out.go:179] * Starting "addons-606715" primary control-plane node in "addons-606715" cluster
I1006 10:52:35.382840 412505 cache.go:123] Beginning downloading kic base image for docker with crio
I1006 10:52:35.383812 412505 out.go:179] * Pulling base image v0.0.48-1759382731-21643 ...
I1006 10:52:35.384682 412505 preload.go:183] Checking if preload exists for k8s version v1.34.1 and runtime crio
I1006 10:52:35.384715 412505 preload.go:198] Found local preload: /home/jenkins/minikube-integration/21699-407371/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.1-cri-o-overlay-amd64.tar.lz4
I1006 10:52:35.384727 412505 cache.go:58] Caching tarball of preloaded images
I1006 10:52:35.384812 412505 preload.go:233] Found /home/jenkins/minikube-integration/21699-407371/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.1-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I1006 10:52:35.384804 412505 image.go:81] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d in local docker daemon
I1006 10:52:35.384827 412505 cache.go:61] Finished verifying existence of preloaded tar for v1.34.1 on crio
I1006 10:52:35.385252 412505 profile.go:143] Saving config to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/config.json ...
I1006 10:52:35.385283 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/config.json: {Name:mkf0fe6564644d16732f44328b45f18a0077625b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:52:35.400788 412505 cache.go:152] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d to local cache
I1006 10:52:35.400899 412505 image.go:65] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d in local cache directory
I1006 10:52:35.400916 412505 image.go:68] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d in local cache directory, skipping pull
I1006 10:52:35.400923 412505 image.go:137] gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d exists in cache, skipping pull
I1006 10:52:35.400946 412505 cache.go:155] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d as a tarball
I1006 10:52:35.400954 412505 cache.go:165] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d from local cache
I1006 10:52:48.741427 412505 cache.go:167] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d from cached tarball
I1006 10:52:48.741478 412505 cache.go:232] Successfully downloaded all kic artifacts
I1006 10:52:48.741524 412505 start.go:360] acquireMachinesLock for addons-606715: {Name:mk2e26986becf76c2e9fc627eb91d2acb0136c8e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1006 10:52:48.741661 412505 start.go:364] duration metric: took 105.326µs to acquireMachinesLock for "addons-606715"
I1006 10:52:48.741699 412505 start.go:93] Provisioning new machine with config: &{Name:addons-606715 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d Memory:4096 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.1 ClusterName:addons-606715 Namespace:default APIServerHAVIP: APIServerName:min
ikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.34.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath:
SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.34.1 ContainerRuntime:crio ControlPlane:true Worker:true}
I1006 10:52:48.741900 412505 start.go:125] createHost starting for "" (driver="docker")
I1006 10:52:48.743685 412505 out.go:252] * Creating docker container (CPUs=2, Memory=4096MB) ...
I1006 10:52:48.743997 412505 start.go:159] libmachine.API.Create for "addons-606715" (driver="docker")
I1006 10:52:48.744037 412505 client.go:168] LocalClient.Create starting
I1006 10:52:48.744139 412505 main.go:141] libmachine: Creating CA: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem
I1006 10:52:48.882473 412505 main.go:141] libmachine: Creating client certificate: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/cert.pem
I1006 10:52:48.990034 412505 cli_runner.go:164] Run: docker network inspect addons-606715 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W1006 10:52:49.007145 412505 cli_runner.go:211] docker network inspect addons-606715 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I1006 10:52:49.007240 412505 network_create.go:284] running [docker network inspect addons-606715] to gather additional debugging logs...
I1006 10:52:49.007267 412505 cli_runner.go:164] Run: docker network inspect addons-606715
W1006 10:52:49.023773 412505 cli_runner.go:211] docker network inspect addons-606715 returned with exit code 1
I1006 10:52:49.023799 412505 network_create.go:287] error running [docker network inspect addons-606715]: docker network inspect addons-606715: exit status 1
stdout:
[]
stderr:
Error response from daemon: network addons-606715 not found
I1006 10:52:49.023817 412505 network_create.go:289] output of [docker network inspect addons-606715]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network addons-606715 not found
** /stderr **
I1006 10:52:49.023954 412505 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1006 10:52:49.041257 412505 network.go:209] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I1006 10:52:49.042718 412505 network.go:209] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I1006 10:52:49.044106 412505 network.go:209] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I1006 10:52:49.045047 412505 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0016fe3c0}
I1006 10:52:49.045078 412505 network_create.go:124] attempt to create docker network addons-606715 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I1006 10:52:49.045131 412505 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=addons-606715 addons-606715
I1006 10:52:49.103468 412505 network_create.go:108] docker network addons-606715 192.168.76.0/24 created
I1006 10:52:49.103504 412505 kic.go:121] calculated static IP "192.168.76.2" for the "addons-606715" container
I1006 10:52:49.103575 412505 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I1006 10:52:49.121908 412505 cli_runner.go:164] Run: docker volume create addons-606715 --label name.minikube.sigs.k8s.io=addons-606715 --label created_by.minikube.sigs.k8s.io=true
I1006 10:52:49.142000 412505 oci.go:103] Successfully created a docker volume addons-606715
I1006 10:52:49.142093 412505 cli_runner.go:164] Run: docker run --rm --name addons-606715-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-606715 --entrypoint /usr/bin/test -v addons-606715:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d -d /var/lib
I1006 10:52:52.448995 412505 cli_runner.go:217] Completed: docker run --rm --name addons-606715-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-606715 --entrypoint /usr/bin/test -v addons-606715:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d -d /var/lib: (3.306819563s)
I1006 10:52:52.449026 412505 oci.go:107] Successfully prepared a docker volume addons-606715
I1006 10:52:52.449054 412505 preload.go:183] Checking if preload exists for k8s version v1.34.1 and runtime crio
I1006 10:52:52.449085 412505 kic.go:194] Starting extracting preloaded images to volume ...
I1006 10:52:52.449156 412505 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21699-407371/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.1-cri-o-overlay-amd64.tar.lz4:/preloaded.tar:ro -v addons-606715:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d -I lz4 -xf /preloaded.tar -C /extractDir
I1006 10:52:56.776250 412505 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21699-407371/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.1-cri-o-overlay-amd64.tar.lz4:/preloaded.tar:ro -v addons-606715:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d -I lz4 -xf /preloaded.tar -C /extractDir: (4.327032677s)
I1006 10:52:56.776286 412505 kic.go:203] duration metric: took 4.327197094s to extract preloaded images to volume ...
W1006 10:52:56.776390 412505 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
W1006 10:52:56.776439 412505 oci.go:252] Your kernel does not support CPU cfs period/quota or the cgroup is not mounted.
I1006 10:52:56.776478 412505 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I1006 10:52:56.829675 412505 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname addons-606715 --name addons-606715 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-606715 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=addons-606715 --network addons-606715 --ip 192.168.76.2 --volume addons-606715:/var --security-opt apparmor=unconfined --memory=4096mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d
I1006 10:52:57.113503 412505 cli_runner.go:164] Run: docker container inspect addons-606715 --format={{.State.Running}}
I1006 10:52:57.131442 412505 cli_runner.go:164] Run: docker container inspect addons-606715 --format={{.State.Status}}
I1006 10:52:57.150051 412505 cli_runner.go:164] Run: docker exec addons-606715 stat /var/lib/dpkg/alternatives/iptables
I1006 10:52:57.199365 412505 oci.go:144] the created container "addons-606715" has a running status.
I1006 10:52:57.199405 412505 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa...
I1006 10:52:57.679235 412505 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I1006 10:52:57.704352 412505 cli_runner.go:164] Run: docker container inspect addons-606715 --format={{.State.Status}}
I1006 10:52:57.723477 412505 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I1006 10:52:57.723500 412505 kic_runner.go:114] Args: [docker exec --privileged addons-606715 chown docker:docker /home/docker/.ssh/authorized_keys]
I1006 10:52:57.764707 412505 cli_runner.go:164] Run: docker container inspect addons-606715 --format={{.State.Status}}
I1006 10:52:57.781907 412505 machine.go:93] provisionDockerMachine start ...
I1006 10:52:57.782013 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:57.798477 412505 main.go:141] libmachine: Using SSH client type: native
I1006 10:52:57.798725 412505 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 32768 <nil> <nil>}
I1006 10:52:57.798741 412505 main.go:141] libmachine: About to run SSH command:
hostname
I1006 10:52:57.941472 412505 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-606715
I1006 10:52:57.941511 412505 ubuntu.go:182] provisioning hostname "addons-606715"
I1006 10:52:57.941604 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:57.959597 412505 main.go:141] libmachine: Using SSH client type: native
I1006 10:52:57.959873 412505 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 32768 <nil> <nil>}
I1006 10:52:57.959892 412505 main.go:141] libmachine: About to run SSH command:
sudo hostname addons-606715 && echo "addons-606715" | sudo tee /etc/hostname
I1006 10:52:58.111518 412505 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-606715
I1006 10:52:58.111600 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:58.129298 412505 main.go:141] libmachine: Using SSH client type: native
I1006 10:52:58.129588 412505 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 32768 <nil> <nil>}
I1006 10:52:58.129613 412505 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\saddons-606715' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-606715/g' /etc/hosts;
else
echo '127.0.1.1 addons-606715' | sudo tee -a /etc/hosts;
fi
fi
I1006 10:52:58.270358 412505 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1006 10:52:58.270388 412505 ubuntu.go:188] set auth options {CertDir:/home/jenkins/minikube-integration/21699-407371/.minikube CaCertPath:/home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/21699-407371/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/21699-407371/.minikube}
I1006 10:52:58.270446 412505 ubuntu.go:190] setting up certificates
I1006 10:52:58.270462 412505 provision.go:84] configureAuth start
I1006 10:52:58.270521 412505 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-606715
I1006 10:52:58.287344 412505 provision.go:143] copyHostCerts
I1006 10:52:58.287445 412505 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/21699-407371/.minikube/ca.pem (1082 bytes)
I1006 10:52:58.287596 412505 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/21699-407371/.minikube/cert.pem (1123 bytes)
I1006 10:52:58.287699 412505 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/21699-407371/.minikube/key.pem (1675 bytes)
I1006 10:52:58.287788 412505 provision.go:117] generating server cert: /home/jenkins/minikube-integration/21699-407371/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca-key.pem org=jenkins.addons-606715 san=[127.0.0.1 192.168.76.2 addons-606715 localhost minikube]
I1006 10:52:58.417628 412505 provision.go:177] copyRemoteCerts
I1006 10:52:58.417715 412505 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1006 10:52:58.417768 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:58.434975 412505 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:32768 SSHKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa Username:docker}
I1006 10:52:58.535190 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I1006 10:52:58.553532 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I1006 10:52:58.569741 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I1006 10:52:58.585704 412505 provision.go:87] duration metric: took 315.226446ms to configureAuth
I1006 10:52:58.585726 412505 ubuntu.go:206] setting minikube options for container-runtime
I1006 10:52:58.585876 412505 config.go:182] Loaded profile config "addons-606715": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.34.1
I1006 10:52:58.585993 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:58.605091 412505 main.go:141] libmachine: Using SSH client type: native
I1006 10:52:58.605319 412505 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 32768 <nil> <nil>}
I1006 10:52:58.605340 412505 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I1006 10:52:58.858098 412505 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I1006 10:52:58.858131 412505 machine.go:96] duration metric: took 1.07619421s to provisionDockerMachine
I1006 10:52:58.858142 412505 client.go:171] duration metric: took 10.114094364s to LocalClient.Create
I1006 10:52:58.858165 412505 start.go:167] duration metric: took 10.1141698s to libmachine.API.Create "addons-606715"
I1006 10:52:58.858176 412505 start.go:293] postStartSetup for "addons-606715" (driver="docker")
I1006 10:52:58.858190 412505 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1006 10:52:58.858277 412505 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1006 10:52:58.858332 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:58.875659 412505 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:32768 SSHKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa Username:docker}
I1006 10:52:58.979051 412505 ssh_runner.go:195] Run: cat /etc/os-release
I1006 10:52:58.982649 412505 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1006 10:52:58.982674 412505 info.go:137] Remote host: Debian GNU/Linux 12 (bookworm)
I1006 10:52:58.982687 412505 filesync.go:126] Scanning /home/jenkins/minikube-integration/21699-407371/.minikube/addons for local assets ...
I1006 10:52:58.982752 412505 filesync.go:126] Scanning /home/jenkins/minikube-integration/21699-407371/.minikube/files for local assets ...
I1006 10:52:58.982781 412505 start.go:296] duration metric: took 124.597217ms for postStartSetup
I1006 10:52:58.983262 412505 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-606715
I1006 10:52:59.000640 412505 profile.go:143] Saving config to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/config.json ...
I1006 10:52:59.000906 412505 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1006 10:52:59.000948 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:59.017610 412505 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:32768 SSHKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa Username:docker}
I1006 10:52:59.116521 412505 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1006 10:52:59.121219 412505 start.go:128] duration metric: took 10.379284232s to createHost
I1006 10:52:59.121247 412505 start.go:83] releasing machines lock for "addons-606715", held for 10.37957005s
I1006 10:52:59.121322 412505 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-606715
I1006 10:52:59.138120 412505 ssh_runner.go:195] Run: cat /version.json
I1006 10:52:59.138171 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:59.138199 412505 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I1006 10:52:59.138285 412505 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-606715
I1006 10:52:59.156036 412505 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:32768 SSHKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa Username:docker}
I1006 10:52:59.156370 412505 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:32768 SSHKeyPath:/home/jenkins/minikube-integration/21699-407371/.minikube/machines/addons-606715/id_rsa Username:docker}
I1006 10:52:59.254831 412505 ssh_runner.go:195] Run: systemctl --version
I1006 10:52:59.306949 412505 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I1006 10:52:59.342497 412505 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W1006 10:52:59.347545 412505 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I1006 10:52:59.347622 412505 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I1006 10:52:59.374234 412505 cni.go:262] disabled [/etc/cni/net.d/10-crio-bridge.conflist.disabled, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I1006 10:52:59.374263 412505 start.go:495] detecting cgroup driver to use...
I1006 10:52:59.374296 412505 detect.go:190] detected "systemd" cgroup driver on host os
I1006 10:52:59.374337 412505 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I1006 10:52:59.391239 412505 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I1006 10:52:59.404033 412505 docker.go:218] disabling cri-docker service (if available) ...
I1006 10:52:59.404109 412505 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I1006 10:52:59.420833 412505 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I1006 10:52:59.438519 412505 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I1006 10:52:59.524297 412505 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I1006 10:52:59.608785 412505 docker.go:234] disabling docker service ...
I1006 10:52:59.608846 412505 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I1006 10:52:59.628161 412505 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I1006 10:52:59.641114 412505 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I1006 10:52:59.723580 412505 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I1006 10:52:59.804484 412505 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 10:52:59.817793 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I1006 10:52:59.832199 412505 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.10.1" pause image...
I1006 10:52:59.832283 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.10.1"|' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.842470 412505 crio.go:70] configuring cri-o to use "systemd" as cgroup driver...
I1006 10:52:59.842554 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "systemd"|' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.851411 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.859999 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.868831 412505 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I1006 10:52:59.876917 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *"net.ipv4.ip_unprivileged_port_start=.*"/d' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.885752 412505 ssh_runner.go:195] Run: sh -c "sudo grep -q "^ *default_sysctls" /etc/crio/crio.conf.d/02-crio.conf || sudo sed -i '/conmon_cgroup = .*/a default_sysctls = \[\n\]' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.899750 412505 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^default_sysctls *= *\[|&\n "net.ipv4.ip_unprivileged_port_start=0",|' /etc/crio/crio.conf.d/02-crio.conf"
I1006 10:52:59.908900 412505 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I1006 10:52:59.916440 412505 crio.go:166] couldn't verify netfilter by "sudo sysctl net.bridge.bridge-nf-call-iptables" which might be okay. error: sudo sysctl net.bridge.bridge-nf-call-iptables: Process exited with status 1
stdout:
stderr:
sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-iptables: No such file or directory
I1006 10:52:59.916497 412505 ssh_runner.go:195] Run: sudo modprobe br_netfilter
I1006 10:52:59.929093 412505 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I1006 10:52:59.937662 412505 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 10:53:00.017177 412505 ssh_runner.go:195] Run: sudo systemctl restart crio
I1006 10:53:00.129523 412505 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I1006 10:53:00.129625 412505 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I1006 10:53:00.134117 412505 start.go:563] Will wait 60s for crictl version
I1006 10:53:00.134171 412505 ssh_runner.go:195] Run: which crictl
I1006 10:53:00.138046 412505 ssh_runner.go:195] Run: sudo /usr/local/bin/crictl version
I1006 10:53:00.163638 412505 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.34.1
RuntimeApiVersion: v1
I1006 10:53:00.163740 412505 ssh_runner.go:195] Run: crio --version
I1006 10:53:00.192800 412505 ssh_runner.go:195] Run: crio --version
I1006 10:53:00.223296 412505 out.go:179] * Preparing Kubernetes v1.34.1 on CRI-O 1.34.1 ...
I1006 10:53:00.224417 412505 cli_runner.go:164] Run: docker network inspect addons-606715 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1006 10:53:00.241846 412505 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I1006 10:53:00.246037 412505 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1006 10:53:00.256245 412505 kubeadm.go:883] updating cluster {Name:addons-606715 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d Memory:4096 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.1 ClusterName:addons-606715 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNa
mes:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketV
MnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I1006 10:53:00.256368 412505 preload.go:183] Checking if preload exists for k8s version v1.34.1 and runtime crio
I1006 10:53:00.256422 412505 ssh_runner.go:195] Run: sudo crictl images --output json
I1006 10:53:00.287736 412505 crio.go:514] all images are preloaded for cri-o runtime.
I1006 10:53:00.287757 412505 crio.go:433] Images already preloaded, skipping extraction
I1006 10:53:00.287806 412505 ssh_runner.go:195] Run: sudo crictl images --output json
I1006 10:53:00.313115 412505 crio.go:514] all images are preloaded for cri-o runtime.
I1006 10:53:00.313138 412505 cache_images.go:85] Images are preloaded, skipping loading
I1006 10:53:00.313145 412505 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.34.1 crio true true} ...
I1006 10:53:00.313267 412505 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.34.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=addons-606715 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.34.1 ClusterName:addons-606715 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I1006 10:53:00.313333 412505 ssh_runner.go:195] Run: crio config
I1006 10:53:00.360121 412505 cni.go:84] Creating CNI manager for ""
I1006 10:53:00.360146 412505 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I1006 10:53:00.360167 412505 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I1006 10:53:00.360230 412505 kubeadm.go:189] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.34.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-606715 NodeName:addons-606715 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:systemd ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernet
es/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I1006 10:53:00.360352 412505 kubeadm.go:195] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/crio/crio.sock
name: "addons-606715"
kubeletExtraArgs:
- name: "node-ip"
value: "192.168.76.2"
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
- name: "enable-admission-plugins"
value: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
- name: "allocate-node-cidrs"
value: "true"
- name: "leader-elect"
value: "false"
scheduler:
extraArgs:
- name: "leader-elect"
value: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
kubernetesVersion: v1.34.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: systemd
containerRuntimeEndpoint: unix:///var/run/crio/crio.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I1006 10:53:00.360413 412505 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.34.1
I1006 10:53:00.368874 412505 binaries.go:44] Found k8s binaries, skipping transfer
I1006 10:53:00.368934 412505 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I1006 10:53:00.376778 412505 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (363 bytes)
I1006 10:53:00.389712 412505 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I1006 10:53:00.405438 412505 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2209 bytes)
I1006 10:53:00.417721 412505 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I1006 10:53:00.421410 412505 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1006 10:53:00.431644 412505 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 10:53:00.509520 412505 ssh_runner.go:195] Run: sudo systemctl start kubelet
I1006 10:53:00.531774 412505 certs.go:69] Setting up /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715 for IP: 192.168.76.2
I1006 10:53:00.531796 412505 certs.go:195] generating shared ca certs ...
I1006 10:53:00.531818 412505 certs.go:227] acquiring lock for ca certs: {Name:mkd159a1e314442ea4dbe67c449a296929269e78 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:00.531953 412505 certs.go:241] generating "minikubeCA" ca cert: /home/jenkins/minikube-integration/21699-407371/.minikube/ca.key
I1006 10:53:00.695651 412505 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21699-407371/.minikube/ca.crt ...
I1006 10:53:00.695686 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/ca.crt: {Name:mk3b80ef03f8dc31044867e73f94bb82df0f0fd5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:00.695893 412505 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21699-407371/.minikube/ca.key ...
I1006 10:53:00.695912 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/ca.key: {Name:mk6df871f87c1f3cdebd8bcbf056e74f1f0f0c77 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:00.696029 412505 certs.go:241] generating "proxyClientCA" ca cert: /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.key
I1006 10:53:00.973059 412505 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.crt ...
I1006 10:53:00.973097 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.crt: {Name:mke96a53ea9513f93b6baef436e810f8678419c6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:00.973326 412505 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.key ...
I1006 10:53:00.973348 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.key: {Name:mk3b69f2acf524fc63ca51fa9644718d77703977 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:00.974258 412505 certs.go:257] generating profile certs ...
I1006 10:53:00.974363 412505 certs.go:364] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.key
I1006 10:53:00.974389 412505 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.crt with IP's: []
I1006 10:53:01.301815 412505 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.crt ...
I1006 10:53:01.301849 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.crt: {Name:mk2d468e90911f9eaa8515479600703113af11f0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.302693 412505 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.key ...
I1006 10:53:01.302716 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/client.key: {Name:mk70a6ee2d5e66a50530284551619c9eb8fcbaf8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.302830 412505 certs.go:364] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key.e6283769
I1006 10:53:01.302856 412505 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt.e6283769 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.76.2]
I1006 10:53:01.331716 412505 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt.e6283769 ...
I1006 10:53:01.331749 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt.e6283769: {Name:mk3bc0010afe8f2d28ae8ce9e6ad8add575d5940 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.331935 412505 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key.e6283769 ...
I1006 10:53:01.331955 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key.e6283769: {Name:mkc4223cf97790a27ad839ce2e0e842bc8d0f359 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.332731 412505 certs.go:382] copying /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt.e6283769 -> /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt
I1006 10:53:01.332843 412505 certs.go:386] copying /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key.e6283769 -> /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key
I1006 10:53:01.332921 412505 certs.go:364] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.key
I1006 10:53:01.332948 412505 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.crt with IP's: []
I1006 10:53:01.406010 412505 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.crt ...
I1006 10:53:01.406051 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.crt: {Name:mk55afaf27e2dacce8eabd27c375db12d8626848 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.406279 412505 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.key ...
I1006 10:53:01.406305 412505 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.key: {Name:mkdddc96478e68a3efc8b417274e9697120faabc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1006 10:53:01.406515 412505 certs.go:484] found cert: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca-key.pem (1675 bytes)
I1006 10:53:01.406569 412505 certs.go:484] found cert: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/ca.pem (1082 bytes)
I1006 10:53:01.406604 412505 certs.go:484] found cert: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/cert.pem (1123 bytes)
I1006 10:53:01.406647 412505 certs.go:484] found cert: /home/jenkins/minikube-integration/21699-407371/.minikube/certs/key.pem (1675 bytes)
I1006 10:53:01.407327 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I1006 10:53:01.426354 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I1006 10:53:01.443925 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I1006 10:53:01.460621 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I1006 10:53:01.477431 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1419 bytes)
I1006 10:53:01.493971 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I1006 10:53:01.510991 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I1006 10:53:01.527615 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/profiles/addons-606715/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I1006 10:53:01.544479 412505 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21699-407371/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I1006 10:53:01.563291 412505 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I1006 10:53:01.575523 412505 ssh_runner.go:195] Run: openssl version
I1006 10:53:01.581675 412505 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I1006 10:53:01.592577 412505 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I1006 10:53:01.596136 412505 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Oct 6 10:53 /usr/share/ca-certificates/minikubeCA.pem
I1006 10:53:01.596221 412505 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I1006 10:53:01.630445 412505 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I1006 10:53:01.639517 412505 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I1006 10:53:01.643285 412505 certs.go:400] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I1006 10:53:01.643339 412505 kubeadm.go:400] StartCluster: {Name:addons-606715 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.48-1759382731-21643@sha256:ca1b4db171879edd6bbb9546a4b1afac2eb5be94a0f5528496e62d2ff99de37d Memory:4096 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.1 ClusterName:addons-606715 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames
:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMne
tClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1006 10:53:01.643421 412505 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I1006 10:53:01.643463 412505 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I1006 10:53:01.671522 412505 cri.go:89] found id: ""
I1006 10:53:01.671608 412505 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I1006 10:53:01.680576 412505 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I1006 10:53:01.688546 412505 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I1006 10:53:01.688601 412505 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I1006 10:53:01.696725 412505 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I1006 10:53:01.696742 412505 kubeadm.go:157] found existing configuration files:
I1006 10:53:01.696787 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I1006 10:53:01.704831 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I1006 10:53:01.704879 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I1006 10:53:01.712159 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I1006 10:53:01.719847 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I1006 10:53:01.719892 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I1006 10:53:01.727025 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I1006 10:53:01.735045 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I1006 10:53:01.735093 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I1006 10:53:01.742412 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I1006 10:53:01.750271 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I1006 10:53:01.750328 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I1006 10:53:01.757855 412505 ssh_runner.go:286] Start: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I1006 10:53:01.797757 412505 kubeadm.go:318] [init] Using Kubernetes version: v1.34.1
I1006 10:53:01.797809 412505 kubeadm.go:318] [preflight] Running pre-flight checks
I1006 10:53:01.819302 412505 kubeadm.go:318] [preflight] The system verification failed. Printing the output from the verification:
I1006 10:53:01.819409 412505 kubeadm.go:318] [0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
I1006 10:53:01.819526 412505 kubeadm.go:318] [0;37mOS[0m: [0;32mLinux[0m
I1006 10:53:01.819642 412505 kubeadm.go:318] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I1006 10:53:01.819723 412505 kubeadm.go:318] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I1006 10:53:01.819801 412505 kubeadm.go:318] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I1006 10:53:01.819901 412505 kubeadm.go:318] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I1006 10:53:01.819964 412505 kubeadm.go:318] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I1006 10:53:01.820044 412505 kubeadm.go:318] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I1006 10:53:01.820124 412505 kubeadm.go:318] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I1006 10:53:01.820189 412505 kubeadm.go:318] [0;37mCGROUPS_IO[0m: [0;32menabled[0m
I1006 10:53:01.878412 412505 kubeadm.go:318] [preflight] Pulling images required for setting up a Kubernetes cluster
I1006 10:53:01.878558 412505 kubeadm.go:318] [preflight] This might take a minute or two, depending on the speed of your internet connection
I1006 10:53:01.878688 412505 kubeadm.go:318] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I1006 10:53:01.886751 412505 kubeadm.go:318] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I1006 10:53:01.888952 412505 out.go:252] - Generating certificates and keys ...
I1006 10:53:01.889069 412505 kubeadm.go:318] [certs] Using existing ca certificate authority
I1006 10:53:01.889191 412505 kubeadm.go:318] [certs] Using existing apiserver certificate and key on disk
I1006 10:53:01.944596 412505 kubeadm.go:318] [certs] Generating "apiserver-kubelet-client" certificate and key
I1006 10:53:02.615718 412505 kubeadm.go:318] [certs] Generating "front-proxy-ca" certificate and key
I1006 10:53:02.691338 412505 kubeadm.go:318] [certs] Generating "front-proxy-client" certificate and key
I1006 10:53:03.270734 412505 kubeadm.go:318] [certs] Generating "etcd/ca" certificate and key
I1006 10:53:03.598849 412505 kubeadm.go:318] [certs] Generating "etcd/server" certificate and key
I1006 10:53:03.598996 412505 kubeadm.go:318] [certs] etcd/server serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I1006 10:53:04.553425 412505 kubeadm.go:318] [certs] Generating "etcd/peer" certificate and key
I1006 10:53:04.553534 412505 kubeadm.go:318] [certs] etcd/peer serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I1006 10:53:04.971453 412505 kubeadm.go:318] [certs] Generating "etcd/healthcheck-client" certificate and key
I1006 10:53:05.025475 412505 kubeadm.go:318] [certs] Generating "apiserver-etcd-client" certificate and key
I1006 10:53:05.241413 412505 kubeadm.go:318] [certs] Generating "sa" key and public key
I1006 10:53:05.241594 412505 kubeadm.go:318] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1006 10:53:05.400366 412505 kubeadm.go:318] [kubeconfig] Writing "admin.conf" kubeconfig file
I1006 10:53:05.700026 412505 kubeadm.go:318] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I1006 10:53:06.023698 412505 kubeadm.go:318] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I1006 10:53:06.354043 412505 kubeadm.go:318] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1006 10:53:06.514251 412505 kubeadm.go:318] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I1006 10:53:06.514878 412505 kubeadm.go:318] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1006 10:53:06.519022 412505 kubeadm.go:318] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I1006 10:53:06.520418 412505 out.go:252] - Booting up control plane ...
I1006 10:53:06.520536 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-apiserver"
I1006 10:53:06.520659 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I1006 10:53:06.521283 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-scheduler"
I1006 10:53:06.537837 412505 kubeadm.go:318] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I1006 10:53:06.537960 412505 kubeadm.go:318] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
I1006 10:53:06.544141 412505 kubeadm.go:318] [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
I1006 10:53:06.544472 412505 kubeadm.go:318] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I1006 10:53:06.544536 412505 kubeadm.go:318] [kubelet-start] Starting the kubelet
I1006 10:53:06.641242 412505 kubeadm.go:318] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I1006 10:53:06.641403 412505 kubeadm.go:318] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I1006 10:53:07.642753 412505 kubeadm.go:318] [kubelet-check] The kubelet is healthy after 1.001563738s
I1006 10:53:07.649156 412505 kubeadm.go:318] [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
I1006 10:53:07.649316 412505 kubeadm.go:318] [control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
I1006 10:53:07.649421 412505 kubeadm.go:318] [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
I1006 10:53:07.649516 412505 kubeadm.go:318] [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
I1006 10:57:07.650031 412505 kubeadm.go:318] [control-plane-check] kube-apiserver is not healthy after 4m0.001051477s
I1006 10:57:07.650332 412505 kubeadm.go:318] [control-plane-check] kube-controller-manager is not healthy after 4m0.000903245s
I1006 10:57:07.650568 412505 kubeadm.go:318] [control-plane-check] kube-scheduler is not healthy after 4m0.001412414s
I1006 10:57:07.650609 412505 kubeadm.go:318]
I1006 10:57:07.650822 412505 kubeadm.go:318] A control plane component may have crashed or exited when started by the container runtime.
I1006 10:57:07.650953 412505 kubeadm.go:318] To troubleshoot, list all containers using your preferred container runtimes CLI.
I1006 10:57:07.651126 412505 kubeadm.go:318] Here is one example how you may list all running Kubernetes containers by using crictl:
I1006 10:57:07.651284 412505 kubeadm.go:318] - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I1006 10:57:07.651350 412505 kubeadm.go:318] Once you have found the failing container, you can inspect its logs with:
I1006 10:57:07.651421 412505 kubeadm.go:318] - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
I1006 10:57:07.651430 412505 kubeadm.go:318]
I1006 10:57:07.654646 412505 kubeadm.go:318] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
I1006 10:57:07.654806 412505 kubeadm.go:318] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I1006 10:57:07.655406 412505 kubeadm.go:318] error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: Get "https://control-plane.minikube.internal:8443/livez?timeout=10s": dial tcp 192.168.76.2:8443: connect: connection refused, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
I1006 10:57:07.655547 412505 kubeadm.go:318] To see the stack trace of this error execute with --v=5 or higher
W1006 10:57:07.655695 412505 out.go:285] ! initialization failed, will try again: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001563738s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001051477s
[control-plane-check] kube-controller-manager is not healthy after 4m0.000903245s
[control-plane-check] kube-scheduler is not healthy after 4m0.001412414s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: Get "https://control-plane.minikube.internal:8443/livez?timeout=10s": dial tcp 192.168.76.2:8443: connect: connection refused, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [addons-606715 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001563738s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001051477s
[control-plane-check] kube-controller-manager is not healthy after 4m0.000903245s
[control-plane-check] kube-scheduler is not healthy after 4m0.001412414s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: Get "https://control-plane.minikube.internal:8443/livez?timeout=10s": dial tcp 192.168.76.2:8443: connect: connection refused, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
I1006 10:57:07.655771 412505 ssh_runner.go:195] Run: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm reset --cri-socket /var/run/crio/crio.sock --force"
I1006 10:57:08.106955 412505 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I1006 10:57:08.119705 412505 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I1006 10:57:08.119761 412505 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I1006 10:57:08.127814 412505 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I1006 10:57:08.127831 412505 kubeadm.go:157] found existing configuration files:
I1006 10:57:08.127888 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I1006 10:57:08.135453 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I1006 10:57:08.135535 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I1006 10:57:08.142642 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I1006 10:57:08.149897 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I1006 10:57:08.149958 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I1006 10:57:08.156809 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I1006 10:57:08.164103 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I1006 10:57:08.164152 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I1006 10:57:08.171326 412505 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I1006 10:57:08.178845 412505 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I1006 10:57:08.178891 412505 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I1006 10:57:08.186832 412505 ssh_runner.go:286] Start: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I1006 10:57:08.224018 412505 kubeadm.go:318] [init] Using Kubernetes version: v1.34.1
I1006 10:57:08.224098 412505 kubeadm.go:318] [preflight] Running pre-flight checks
I1006 10:57:08.243541 412505 kubeadm.go:318] [preflight] The system verification failed. Printing the output from the verification:
I1006 10:57:08.243659 412505 kubeadm.go:318] [0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
I1006 10:57:08.243711 412505 kubeadm.go:318] [0;37mOS[0m: [0;32mLinux[0m
I1006 10:57:08.243789 412505 kubeadm.go:318] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I1006 10:57:08.243878 412505 kubeadm.go:318] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I1006 10:57:08.243961 412505 kubeadm.go:318] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I1006 10:57:08.244028 412505 kubeadm.go:318] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I1006 10:57:08.244094 412505 kubeadm.go:318] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I1006 10:57:08.244187 412505 kubeadm.go:318] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I1006 10:57:08.244281 412505 kubeadm.go:318] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I1006 10:57:08.244340 412505 kubeadm.go:318] [0;37mCGROUPS_IO[0m: [0;32menabled[0m
I1006 10:57:08.303700 412505 kubeadm.go:318] [preflight] Pulling images required for setting up a Kubernetes cluster
I1006 10:57:08.303801 412505 kubeadm.go:318] [preflight] This might take a minute or two, depending on the speed of your internet connection
I1006 10:57:08.303932 412505 kubeadm.go:318] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I1006 10:57:08.310526 412505 kubeadm.go:318] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I1006 10:57:08.314159 412505 out.go:252] - Generating certificates and keys ...
I1006 10:57:08.314284 412505 kubeadm.go:318] [certs] Using existing ca certificate authority
I1006 10:57:08.314380 412505 kubeadm.go:318] [certs] Using existing apiserver certificate and key on disk
I1006 10:57:08.314501 412505 kubeadm.go:318] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I1006 10:57:08.314592 412505 kubeadm.go:318] [certs] Using existing front-proxy-ca certificate authority
I1006 10:57:08.314686 412505 kubeadm.go:318] [certs] Using existing front-proxy-client certificate and key on disk
I1006 10:57:08.314759 412505 kubeadm.go:318] [certs] Using existing etcd/ca certificate authority
I1006 10:57:08.314833 412505 kubeadm.go:318] [certs] Using existing etcd/server certificate and key on disk
I1006 10:57:08.314926 412505 kubeadm.go:318] [certs] Using existing etcd/peer certificate and key on disk
I1006 10:57:08.315025 412505 kubeadm.go:318] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I1006 10:57:08.315131 412505 kubeadm.go:318] [certs] Using existing apiserver-etcd-client certificate and key on disk
I1006 10:57:08.315217 412505 kubeadm.go:318] [certs] Using the existing "sa" key
I1006 10:57:08.315303 412505 kubeadm.go:318] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1006 10:57:08.573952 412505 kubeadm.go:318] [kubeconfig] Writing "admin.conf" kubeconfig file
I1006 10:57:08.738120 412505 kubeadm.go:318] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I1006 10:57:09.293476 412505 kubeadm.go:318] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I1006 10:57:09.582813 412505 kubeadm.go:318] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1006 10:57:09.934283 412505 kubeadm.go:318] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I1006 10:57:09.934576 412505 kubeadm.go:318] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1006 10:57:09.936759 412505 kubeadm.go:318] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I1006 10:57:09.939667 412505 out.go:252] - Booting up control plane ...
I1006 10:57:09.939799 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-apiserver"
I1006 10:57:09.939882 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I1006 10:57:09.939969 412505 kubeadm.go:318] [control-plane] Creating static Pod manifest for "kube-scheduler"
I1006 10:57:09.952741 412505 kubeadm.go:318] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I1006 10:57:09.952851 412505 kubeadm.go:318] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
I1006 10:57:09.959430 412505 kubeadm.go:318] [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
I1006 10:57:09.960494 412505 kubeadm.go:318] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I1006 10:57:09.960557 412505 kubeadm.go:318] [kubelet-start] Starting the kubelet
I1006 10:57:10.065065 412505 kubeadm.go:318] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I1006 10:57:10.065200 412505 kubeadm.go:318] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I1006 10:57:11.066805 412505 kubeadm.go:318] [kubelet-check] The kubelet is healthy after 1.001852639s
I1006 10:57:11.069801 412505 kubeadm.go:318] [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
I1006 10:57:11.069911 412505 kubeadm.go:318] [control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
I1006 10:57:11.070020 412505 kubeadm.go:318] [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
I1006 10:57:11.070103 412505 kubeadm.go:318] [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
I1006 11:01:11.071903 412505 kubeadm.go:318] [control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
I1006 11:01:11.072368 412505 kubeadm.go:318] [control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
I1006 11:01:11.072562 412505 kubeadm.go:318] [control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
I1006 11:01:11.072600 412505 kubeadm.go:318]
I1006 11:01:11.072814 412505 kubeadm.go:318] A control plane component may have crashed or exited when started by the container runtime.
I1006 11:01:11.073033 412505 kubeadm.go:318] To troubleshoot, list all containers using your preferred container runtimes CLI.
I1006 11:01:11.073285 412505 kubeadm.go:318] Here is one example how you may list all running Kubernetes containers by using crictl:
I1006 11:01:11.073503 412505 kubeadm.go:318] - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
I1006 11:01:11.073688 412505 kubeadm.go:318] Once you have found the failing container, you can inspect its logs with:
I1006 11:01:11.073885 412505 kubeadm.go:318] - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
I1006 11:01:11.073903 412505 kubeadm.go:318]
I1006 11:01:11.076298 412505 kubeadm.go:318] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
I1006 11:01:11.076401 412505 kubeadm.go:318] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I1006 11:01:11.076943 412505 kubeadm.go:318] error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
I1006 11:01:11.077046 412505 kubeadm.go:318] To see the stack trace of this error execute with --v=5 or higher
I1006 11:01:11.077135 412505 kubeadm.go:402] duration metric: took 8m9.433800071s to StartCluster
I1006 11:01:11.077252 412505 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I1006 11:01:11.077349 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I1006 11:01:11.106108 412505 cri.go:89] found id: ""
I1006 11:01:11.106162 412505 logs.go:282] 0 containers: []
W1006 11:01:11.106182 412505 logs.go:284] No container was found matching "kube-apiserver"
I1006 11:01:11.106193 412505 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I1006 11:01:11.106272 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I1006 11:01:11.134406 412505 cri.go:89] found id: ""
I1006 11:01:11.134432 412505 logs.go:282] 0 containers: []
W1006 11:01:11.134440 412505 logs.go:284] No container was found matching "etcd"
I1006 11:01:11.134448 412505 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I1006 11:01:11.134508 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I1006 11:01:11.160587 412505 cri.go:89] found id: ""
I1006 11:01:11.160619 412505 logs.go:282] 0 containers: []
W1006 11:01:11.160630 412505 logs.go:284] No container was found matching "coredns"
I1006 11:01:11.160639 412505 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I1006 11:01:11.160703 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I1006 11:01:11.187831 412505 cri.go:89] found id: ""
I1006 11:01:11.187858 412505 logs.go:282] 0 containers: []
W1006 11:01:11.187867 412505 logs.go:284] No container was found matching "kube-scheduler"
I1006 11:01:11.187874 412505 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I1006 11:01:11.187939 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I1006 11:01:11.214783 412505 cri.go:89] found id: ""
I1006 11:01:11.214816 412505 logs.go:282] 0 containers: []
W1006 11:01:11.214830 412505 logs.go:284] No container was found matching "kube-proxy"
I1006 11:01:11.214840 412505 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I1006 11:01:11.214906 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I1006 11:01:11.241813 412505 cri.go:89] found id: ""
I1006 11:01:11.241843 412505 logs.go:282] 0 containers: []
W1006 11:01:11.241854 412505 logs.go:284] No container was found matching "kube-controller-manager"
I1006 11:01:11.241860 412505 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I1006 11:01:11.241926 412505 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I1006 11:01:11.269001 412505 cri.go:89] found id: ""
I1006 11:01:11.269032 412505 logs.go:282] 0 containers: []
W1006 11:01:11.269043 412505 logs.go:284] No container was found matching "kindnet"
I1006 11:01:11.269056 412505 logs.go:123] Gathering logs for kubelet ...
I1006 11:01:11.269068 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I1006 11:01:11.332839 412505 logs.go:123] Gathering logs for dmesg ...
I1006 11:01:11.332873 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1006 11:01:11.345192 412505 logs.go:123] Gathering logs for describe nodes ...
I1006 11:01:11.345236 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W1006 11:01:11.405120 412505 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
E1006 11:01:11.397540 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.398125 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.399757 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.400189 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.401630 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
E1006 11:01:11.397540 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.398125 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.399757 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.400189 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
E1006 11:01:11.401630 2400 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://localhost:8443/api?timeout=32s\": dial tcp [::1]:8443: connect: connection refused"
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I1006 11:01:11.405164 412505 logs.go:123] Gathering logs for CRI-O ...
I1006 11:01:11.405178 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I1006 11:01:11.464361 412505 logs.go:123] Gathering logs for container status ...
I1006 11:01:11.464409 412505 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W1006 11:01:11.494068 412505 out.go:434] Error starting cluster: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001852639s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
[control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
[control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
W1006 11:01:11.494138 412505 out.go:285] *
*
W1006 11:01:11.494239 412505 out.go:285] X Error starting cluster: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001852639s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
[control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
[control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001852639s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
[control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
[control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
W1006 11:01:11.494262 412505 out.go:285] *
*
W1006 11:01:11.496117 412505 out.go:308] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1006 11:01:11.499640 412505 out.go:203]
W1006 11:01:11.500630 412505 out.go:285] X Exiting due to GUEST_START: failed to start node: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001852639s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
[control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
[control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to GUEST_START: failed to start node: wait: sudo /bin/bash -c "env PATH="/var/lib/minikube/binaries/v1.34.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.34.1
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m6.8.0-1041-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_IO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.001852639s
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.76.2:8443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-apiserver is not healthy after 4m0.001293717s
[control-plane-check] kube-controller-manager is not healthy after 4m0.001339855s
[control-plane-check] kube-scheduler is not healthy after 4m0.00139634s
A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'
stderr:
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/6.8.0-1041-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error: error execution phase wait-control-plane: failed while waiting for the control plane to start: [kube-apiserver check failed at https://192.168.76.2:8443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline, kube-controller-manager check failed at https://127.0.0.1:10257/healthz: Get "https://127.0.0.1:10257/healthz": dial tcp 127.0.0.1:10257: connect: connection refused, kube-scheduler check failed at https://127.0.0.1:10259/livez: Get "https://127.0.0.1:10259/livez": dial tcp 127.0.0.1:10259: connect: connection refused]
To see the stack trace of this error execute with --v=5 or higher
W1006 11:01:11.500651 412505 out.go:285] *
*
I1006 11:01:11.501922 412505 out.go:203]
** /stderr **
addons_test.go:110: out/minikube-linux-amd64 start -p addons-606715 --wait=true --memory=4096 --alsologtostderr --addons=registry --addons=registry-creds --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=nvidia-device-plugin --addons=yakd --addons=volcano --addons=amd-gpu-device-plugin --driver=docker --container-runtime=crio --addons=ingress --addons=ingress-dns --addons=storage-provisioner-rancher failed: exit status 80
--- FAIL: TestAddons/Setup (516.36s)