=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-554000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-554000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m19.403639084s)
-- stdout --
* [old-k8s-version-554000] minikube v1.30.1 on Darwin 13.4
- MINIKUBE_LOCATION=15003
- KUBECONFIG=/Users/jenkins/minikube-integration/15003-20351/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15003-20351/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-554000 in cluster old-k8s-version-554000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 24.0.2 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0613 12:39:22.826629 36348 out.go:296] Setting OutFile to fd 1 ...
I0613 12:39:22.826826 36348 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0613 12:39:22.826832 36348 out.go:309] Setting ErrFile to fd 2...
I0613 12:39:22.826836 36348 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0613 12:39:22.826960 36348 root.go:336] Updating PATH: /Users/jenkins/minikube-integration/15003-20351/.minikube/bin
I0613 12:39:22.829196 36348 out.go:303] Setting JSON to false
I0613 12:39:22.860377 36348 start.go:127] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":9533,"bootTime":1686675629,"procs":442,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.4","kernelVersion":"22.5.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W0613 12:39:22.860493 36348 start.go:135] gopshost.Virtualization returned error: not implemented yet
I0613 12:39:22.900656 36348 out.go:177] * [old-k8s-version-554000] minikube v1.30.1 on Darwin 13.4
I0613 12:39:22.921531 36348 notify.go:220] Checking for updates...
I0613 12:39:22.958627 36348 out.go:177] - MINIKUBE_LOCATION=15003
I0613 12:39:23.034713 36348 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15003-20351/kubeconfig
I0613 12:39:23.080772 36348 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0613 12:39:23.155670 36348 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0613 12:39:23.213710 36348 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15003-20351/.minikube
I0613 12:39:23.255589 36348 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0613 12:39:23.277288 36348 config.go:182] Loaded profile config "kubenet-185000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.27.2
I0613 12:39:23.277403 36348 driver.go:373] Setting default libvirt URI to qemu:///system
I0613 12:39:23.340155 36348 docker.go:121] docker version: linux-24.0.2:Docker Desktop 4.20.1 (110738)
I0613 12:39:23.340316 36348 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0613 12:39:23.452424 36348 info.go:266] docker info: {ID:c18f23ef-4e44-410e-b2ce-38db72a58e15 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:58 OomKillDisable:false NGoroutines:66 SystemTime:2023-06-13 19:39:23.436083886 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:7 KernelVersion:5.15.49-linuxkit-pr OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexS
erverAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:3dce8eb055cbb6872793272b4f20ed16117344f8 Expected:3dce8eb055cbb6872793272b4f20ed16117344f8} RuncCommit:{ID:v1.1.7-0-g860f061 Expected:v1.1.7-0-g860f061} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=built
in name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.5] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.18.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.19] map[Name:init Pat
h:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.4] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Command line tool for Docker Scout Vendor:Docker Inc. Version:v0.12.0]] Warnings:<
nil>}}
I0613 12:39:23.476658 36348 out.go:177] * Using the docker driver based on user configuration
I0613 12:39:23.497710 36348 start.go:297] selected driver: docker
I0613 12:39:23.497733 36348 start.go:884] validating driver "docker" against <nil>
I0613 12:39:23.497779 36348 start.go:895] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0613 12:39:23.501776 36348 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0613 12:39:23.595704 36348 info.go:266] docker info: {ID:c18f23ef-4e44-410e-b2ce-38db72a58e15 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:58 OomKillDisable:false NGoroutines:66 SystemTime:2023-06-13 19:39:23.585377238 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:7 KernelVersion:5.15.49-linuxkit-pr OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexS
erverAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:3dce8eb055cbb6872793272b4f20ed16117344f8 Expected:3dce8eb055cbb6872793272b4f20ed16117344f8} RuncCommit:{ID:v1.1.7-0-g860f061 Expected:v1.1.7-0-g860f061} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=built
in name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.5] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.18.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.19] map[Name:init Pat
h:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.4] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Command line tool for Docker Scout Vendor:Docker Inc. Version:v0.12.0]] Warnings:<
nil>}}
I0613 12:39:23.595888 36348 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0613 12:39:23.596094 36348 start_flags.go:919] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0613 12:39:23.617856 36348 out.go:177] * Using Docker Desktop driver with root privileges
I0613 12:39:23.639363 36348 cni.go:84] Creating CNI manager for ""
I0613 12:39:23.639389 36348 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0613 12:39:23.639401 36348 start_flags.go:319] config:
{Name:old-k8s-version-554000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-554000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0613 12:39:23.660421 36348 out.go:177] * Starting control plane node old-k8s-version-554000 in cluster old-k8s-version-554000
I0613 12:39:23.681574 36348 cache.go:122] Beginning downloading kic base image for docker with docker
I0613 12:39:23.703355 36348 out.go:177] * Pulling base image ...
I0613 12:39:23.747529 36348 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0613 12:39:23.747562 36348 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b in local docker daemon
I0613 12:39:23.747640 36348 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0613 12:39:23.747669 36348 cache.go:57] Caching tarball of preloaded images
I0613 12:39:23.747880 36348 preload.go:174] Found /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0613 12:39:23.747898 36348 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0613 12:39:23.748992 36348 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/config.json ...
I0613 12:39:23.749143 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/config.json: {Name:mk355b9596d33ea4a4b0d801dae1dde1e9cdeae9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:23.798636 36348 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b in local docker daemon, skipping pull
I0613 12:39:23.798654 36348 cache.go:145] gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b exists in daemon, skipping load
I0613 12:39:23.798674 36348 cache.go:195] Successfully downloaded all kic artifacts
I0613 12:39:23.798836 36348 start.go:365] acquiring machines lock for old-k8s-version-554000: {Name:mk0a9b1134645f4b38304ff0b8ed03f330d2f839 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0613 12:39:23.799004 36348 start.go:369] acquired machines lock for "old-k8s-version-554000" in 156.441µs
I0613 12:39:23.799035 36348 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-554000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-554000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0613 12:39:23.799109 36348 start.go:125] createHost starting for "" (driver="docker")
I0613 12:39:23.820290 36348 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0613 12:39:23.820600 36348 start.go:159] libmachine.API.Create for "old-k8s-version-554000" (driver="docker")
I0613 12:39:23.820638 36348 client.go:168] LocalClient.Create starting
I0613 12:39:23.820771 36348 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem
I0613 12:39:23.820818 36348 main.go:141] libmachine: Decoding PEM data...
I0613 12:39:23.820842 36348 main.go:141] libmachine: Parsing certificate...
I0613 12:39:23.820937 36348 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/cert.pem
I0613 12:39:23.820975 36348 main.go:141] libmachine: Decoding PEM data...
I0613 12:39:23.820989 36348 main.go:141] libmachine: Parsing certificate...
I0613 12:39:23.821582 36348 cli_runner.go:164] Run: docker network inspect old-k8s-version-554000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0613 12:39:23.871955 36348 cli_runner.go:211] docker network inspect old-k8s-version-554000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0613 12:39:23.872070 36348 network_create.go:281] running [docker network inspect old-k8s-version-554000] to gather additional debugging logs...
I0613 12:39:23.872093 36348 cli_runner.go:164] Run: docker network inspect old-k8s-version-554000
W0613 12:39:23.921673 36348 cli_runner.go:211] docker network inspect old-k8s-version-554000 returned with exit code 1
I0613 12:39:23.921704 36348 network_create.go:284] error running [docker network inspect old-k8s-version-554000]: docker network inspect old-k8s-version-554000: exit status 1
stdout:
[]
stderr:
Error response from daemon: network old-k8s-version-554000 not found
I0613 12:39:23.921731 36348 network_create.go:286] output of [docker network inspect old-k8s-version-554000]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network old-k8s-version-554000 not found
** /stderr **
I0613 12:39:23.921822 36348 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0613 12:39:23.973056 36348 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0613 12:39:23.973397 36348 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00104efb0}
I0613 12:39:23.973413 36348 network_create.go:123] attempt to create docker network old-k8s-version-554000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...
I0613 12:39:23.973518 36348 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-554000 old-k8s-version-554000
W0613 12:39:24.022974 36348 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-554000 old-k8s-version-554000 returned with exit code 1
W0613 12:39:24.023016 36348 network_create.go:148] failed to create docker network old-k8s-version-554000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 65535: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-554000 old-k8s-version-554000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0613 12:39:24.023036 36348 network_create.go:115] failed to create docker network old-k8s-version-554000 192.168.58.0/24, will retry: subnet is taken
I0613 12:39:24.024473 36348 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0613 12:39:24.024792 36348 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0008b69d0}
I0613 12:39:24.024808 36348 network_create.go:123] attempt to create docker network old-k8s-version-554000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...
I0613 12:39:24.024874 36348 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-554000 old-k8s-version-554000
I0613 12:39:24.108269 36348 network_create.go:107] docker network old-k8s-version-554000 192.168.67.0/24 created
I0613 12:39:24.108305 36348 kic.go:117] calculated static IP "192.168.67.2" for the "old-k8s-version-554000" container
I0613 12:39:24.108440 36348 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0613 12:39:24.158070 36348 cli_runner.go:164] Run: docker volume create old-k8s-version-554000 --label name.minikube.sigs.k8s.io=old-k8s-version-554000 --label created_by.minikube.sigs.k8s.io=true
I0613 12:39:24.207400 36348 oci.go:103] Successfully created a docker volume old-k8s-version-554000
I0613 12:39:24.207525 36348 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-554000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-554000 --entrypoint /usr/bin/test -v old-k8s-version-554000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b -d /var/lib
I0613 12:39:24.644585 36348 oci.go:107] Successfully prepared a docker volume old-k8s-version-554000
I0613 12:39:24.644626 36348 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0613 12:39:24.644641 36348 kic.go:190] Starting extracting preloaded images to volume ...
I0613 12:39:24.644756 36348 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-554000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b -I lz4 -xf /preloaded.tar -C /extractDir
I0613 12:39:30.320468 36348 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-554000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b -I lz4 -xf /preloaded.tar -C /extractDir: (5.675755227s)
I0613 12:39:30.320490 36348 kic.go:199] duration metric: took 5.675966 seconds to extract preloaded images to volume
I0613 12:39:30.320620 36348 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0613 12:39:30.424685 36348 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-554000 --name old-k8s-version-554000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-554000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-554000 --network old-k8s-version-554000 --ip 192.168.67.2 --volume old-k8s-version-554000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b
I0613 12:39:30.737732 36348 cli_runner.go:164] Run: docker container inspect old-k8s-version-554000 --format={{.State.Running}}
I0613 12:39:30.804755 36348 cli_runner.go:164] Run: docker container inspect old-k8s-version-554000 --format={{.State.Status}}
I0613 12:39:30.868500 36348 cli_runner.go:164] Run: docker exec old-k8s-version-554000 stat /var/lib/dpkg/alternatives/iptables
I0613 12:39:30.985364 36348 oci.go:144] the created container "old-k8s-version-554000" has a running status.
I0613 12:39:30.985442 36348 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa...
I0613 12:39:31.026497 36348 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0613 12:39:31.096306 36348 cli_runner.go:164] Run: docker container inspect old-k8s-version-554000 --format={{.State.Status}}
I0613 12:39:31.155965 36348 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0613 12:39:31.155997 36348 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-554000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0613 12:39:31.280450 36348 cli_runner.go:164] Run: docker container inspect old-k8s-version-554000 --format={{.State.Status}}
I0613 12:39:31.346719 36348 machine.go:88] provisioning docker machine ...
I0613 12:39:31.346765 36348 ubuntu.go:169] provisioning hostname "old-k8s-version-554000"
I0613 12:39:31.346940 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:31.407111 36348 main.go:141] libmachine: Using SSH client type: native
I0613 12:39:31.407514 36348 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140cac0] 0x140fb60 <nil> [] 0s} 127.0.0.1 59436 <nil> <nil>}
I0613 12:39:31.407529 36348 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-554000 && echo "old-k8s-version-554000" | sudo tee /etc/hostname
I0613 12:39:31.548359 36348 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-554000
I0613 12:39:31.548496 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:31.607671 36348 main.go:141] libmachine: Using SSH client type: native
I0613 12:39:31.608101 36348 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140cac0] 0x140fb60 <nil> [] 0s} 127.0.0.1 59436 <nil> <nil>}
I0613 12:39:31.608117 36348 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-554000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-554000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-554000' | sudo tee -a /etc/hosts;
fi
fi
I0613 12:39:31.732275 36348 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0613 12:39:31.732307 36348 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15003-20351/.minikube CaCertPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15003-20351/.minikube}
I0613 12:39:31.732330 36348 ubuntu.go:177] setting up certificates
I0613 12:39:31.732347 36348 provision.go:83] configureAuth start
I0613 12:39:31.732429 36348 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-554000
I0613 12:39:31.787037 36348 provision.go:138] copyHostCerts
I0613 12:39:31.787140 36348 exec_runner.go:144] found /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.pem, removing ...
I0613 12:39:31.787150 36348 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.pem
I0613 12:39:31.787251 36348 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.pem (1082 bytes)
I0613 12:39:31.787451 36348 exec_runner.go:144] found /Users/jenkins/minikube-integration/15003-20351/.minikube/cert.pem, removing ...
I0613 12:39:31.787458 36348 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/15003-20351/.minikube/cert.pem
I0613 12:39:31.787520 36348 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15003-20351/.minikube/cert.pem (1123 bytes)
I0613 12:39:31.787695 36348 exec_runner.go:144] found /Users/jenkins/minikube-integration/15003-20351/.minikube/key.pem, removing ...
I0613 12:39:31.787702 36348 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/15003-20351/.minikube/key.pem
I0613 12:39:31.787761 36348 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15003-20351/.minikube/key.pem (1679 bytes)
I0613 12:39:31.787913 36348 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-554000 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-554000]
I0613 12:39:31.893572 36348 provision.go:172] copyRemoteCerts
I0613 12:39:31.893641 36348 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0613 12:39:31.893702 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:31.988591 36348 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59436 SSHKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa Username:docker}
I0613 12:39:32.076565 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0613 12:39:32.100284 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0613 12:39:32.125905 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0613 12:39:32.152845 36348 provision.go:86] duration metric: configureAuth took 420.481679ms
I0613 12:39:32.152863 36348 ubuntu.go:193] setting minikube options for container-runtime
I0613 12:39:32.153016 36348 config.go:182] Loaded profile config "old-k8s-version-554000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0613 12:39:32.153103 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:32.206347 36348 main.go:141] libmachine: Using SSH client type: native
I0613 12:39:32.206697 36348 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140cac0] 0x140fb60 <nil> [] 0s} 127.0.0.1 59436 <nil> <nil>}
I0613 12:39:32.206715 36348 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0613 12:39:32.328297 36348 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0613 12:39:32.328314 36348 ubuntu.go:71] root file system type: overlay
I0613 12:39:32.328427 36348 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0613 12:39:32.328524 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:32.382048 36348 main.go:141] libmachine: Using SSH client type: native
I0613 12:39:32.382388 36348 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140cac0] 0x140fb60 <nil> [] 0s} 127.0.0.1 59436 <nil> <nil>}
I0613 12:39:32.382436 36348 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0613 12:39:32.510858 36348 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0613 12:39:32.510959 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:32.567366 36348 main.go:141] libmachine: Using SSH client type: native
I0613 12:39:32.567722 36348 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140cac0] 0x140fb60 <nil> [] 0s} 127.0.0.1 59436 <nil> <nil>}
I0613 12:39:32.567736 36348 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0613 12:39:33.348977 36348 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-05-25 21:51:00.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-06-13 19:39:32.509025763 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0613 12:39:33.349002 36348 machine.go:91] provisioned docker machine in 2.002304747s
I0613 12:39:33.349008 36348 client.go:171] LocalClient.Create took 9.528567044s
I0613 12:39:33.349027 36348 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-554000" took 9.528628144s
I0613 12:39:33.349038 36348 start.go:300] post-start starting for "old-k8s-version-554000" (driver="docker")
I0613 12:39:33.349051 36348 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0613 12:39:33.349118 36348 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0613 12:39:33.349185 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:33.402651 36348 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59436 SSHKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa Username:docker}
I0613 12:39:33.492440 36348 ssh_runner.go:195] Run: cat /etc/os-release
I0613 12:39:33.496916 36348 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0613 12:39:33.496937 36348 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0613 12:39:33.496946 36348 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0613 12:39:33.496953 36348 info.go:137] Remote host: Ubuntu 22.04.2 LTS
I0613 12:39:33.496963 36348 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15003-20351/.minikube/addons for local assets ...
I0613 12:39:33.497044 36348 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15003-20351/.minikube/files for local assets ...
I0613 12:39:33.497226 36348 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15003-20351/.minikube/files/etc/ssl/certs/208002.pem -> 208002.pem in /etc/ssl/certs
I0613 12:39:33.497421 36348 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0613 12:39:33.506280 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/files/etc/ssl/certs/208002.pem --> /etc/ssl/certs/208002.pem (1708 bytes)
I0613 12:39:33.530350 36348 start.go:303] post-start completed in 181.301782ms
I0613 12:39:33.530958 36348 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-554000
I0613 12:39:33.586238 36348 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/config.json ...
I0613 12:39:33.586705 36348 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0613 12:39:33.586775 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:33.643178 36348 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59436 SSHKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa Username:docker}
I0613 12:39:33.731125 36348 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0613 12:39:33.736592 36348 start.go:128] duration metric: createHost completed in 9.937682562s
I0613 12:39:33.736610 36348 start.go:83] releasing machines lock for "old-k8s-version-554000", held for 9.937807347s
I0613 12:39:33.736691 36348 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-554000
I0613 12:39:33.785941 36348 ssh_runner.go:195] Run: cat /version.json
I0613 12:39:33.785962 36348 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0613 12:39:33.786023 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:33.786040 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:33.839913 36348 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59436 SSHKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa Username:docker}
I0613 12:39:33.839972 36348 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59436 SSHKeyPath:/Users/jenkins/minikube-integration/15003-20351/.minikube/machines/old-k8s-version-554000/id_rsa Username:docker}
I0613 12:39:34.027887 36348 ssh_runner.go:195] Run: systemctl --version
I0613 12:39:34.033365 36348 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0613 12:39:34.039021 36348 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0613 12:39:34.062775 36348 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0613 12:39:34.062870 36348 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0613 12:39:34.079354 36348 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0613 12:39:34.095576 36348 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0613 12:39:34.095592 36348 start.go:464] detecting cgroup driver to use...
I0613 12:39:34.095607 36348 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0613 12:39:34.095716 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0613 12:39:34.111561 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.1"|' /etc/containerd/config.toml"
I0613 12:39:34.121867 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0613 12:39:34.131973 36348 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0613 12:39:34.132037 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0613 12:39:34.142305 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0613 12:39:34.152495 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0613 12:39:34.162573 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0613 12:39:34.172501 36348 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0613 12:39:34.182049 36348 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0613 12:39:34.192244 36348 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0613 12:39:34.201176 36348 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0613 12:39:34.210287 36348 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0613 12:39:34.281743 36348 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0613 12:39:34.362720 36348 start.go:464] detecting cgroup driver to use...
I0613 12:39:34.362746 36348 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0613 12:39:34.362829 36348 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0613 12:39:34.375443 36348 cruntime.go:276] skipping containerd shutdown because we are bound to it
I0613 12:39:34.375511 36348 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0613 12:39:34.387345 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0613 12:39:34.404215 36348 ssh_runner.go:195] Run: which cri-dockerd
I0613 12:39:34.411367 36348 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0613 12:39:34.420739 36348 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0613 12:39:34.439551 36348 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0613 12:39:34.544183 36348 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0613 12:39:34.627384 36348 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...
I0613 12:39:34.627399 36348 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0613 12:39:34.645759 36348 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0613 12:39:34.722639 36348 ssh_runner.go:195] Run: sudo systemctl restart docker
I0613 12:39:34.975612 36348 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0613 12:39:35.003641 36348 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0613 12:39:35.092339 36348 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 24.0.2 ...
I0613 12:39:35.092523 36348 cli_runner.go:164] Run: docker exec -t old-k8s-version-554000 dig +short host.docker.internal
I0613 12:39:35.204380 36348 network.go:96] got host ip for mount in container by digging dns: 192.168.65.254
I0613 12:39:35.204498 36348 ssh_runner.go:195] Run: grep 192.168.65.254 host.minikube.internal$ /etc/hosts
I0613 12:39:35.209617 36348 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.254 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0613 12:39:35.221174 36348 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-554000
I0613 12:39:35.270473 36348 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0613 12:39:35.270545 36348 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0613 12:39:35.293584 36348 docker.go:636] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0613 12:39:35.293598 36348 docker.go:642] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I0613 12:39:35.293668 36348 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0613 12:39:35.303257 36348 ssh_runner.go:195] Run: which lz4
I0613 12:39:35.307505 36348 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0613 12:39:35.311933 36348 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0613 12:39:35.311966 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (369789069 bytes)
I0613 12:39:40.184826 36348 docker.go:600] Took 4.877511 seconds to copy over tarball
I0613 12:39:40.184917 36348 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0613 12:39:42.715015 36348 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.530135815s)
I0613 12:39:42.715030 36348 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0613 12:39:42.788660 36348 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0613 12:39:42.798228 36348 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2499 bytes)
I0613 12:39:42.814784 36348 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0613 12:39:42.897252 36348 ssh_runner.go:195] Run: sudo systemctl restart docker
I0613 12:39:43.600943 36348 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0613 12:39:43.623506 36348 docker.go:636] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0613 12:39:43.623519 36348 docker.go:642] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I0613 12:39:43.623527 36348 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.16.0 registry.k8s.io/kube-controller-manager:v1.16.0 registry.k8s.io/kube-scheduler:v1.16.0 registry.k8s.io/kube-proxy:v1.16.0 registry.k8s.io/pause:3.1 registry.k8s.io/etcd:3.3.15-0 registry.k8s.io/coredns:1.6.2 gcr.io/k8s-minikube/storage-provisioner:v5]
I0613 12:39:43.629337 36348 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0613 12:39:43.631132 36348 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.16.0
I0613 12:39:43.631159 36348 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.16.0
I0613 12:39:43.631166 36348 image.go:134] retrieving image: registry.k8s.io/etcd:3.3.15-0
I0613 12:39:43.631172 36348 image.go:134] retrieving image: registry.k8s.io/pause:3.1
I0613 12:39:43.631231 36348 image.go:134] retrieving image: registry.k8s.io/coredns:1.6.2
I0613 12:39:43.631265 36348 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.16.0
I0613 12:39:43.631400 36348 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.16.0
I0613 12:39:43.634937 36348 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0613 12:39:43.637930 36348 image.go:177] daemon lookup for registry.k8s.io/coredns:1.6.2: Error response from daemon: No such image: registry.k8s.io/coredns:1.6.2
I0613 12:39:43.639593 36348 image.go:177] daemon lookup for registry.k8s.io/pause:3.1: Error response from daemon: No such image: registry.k8s.io/pause:3.1
I0613 12:39:43.639657 36348 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.16.0
I0613 12:39:43.639675 36348 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.16.0
I0613 12:39:43.639690 36348 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.16.0
I0613 12:39:43.639694 36348 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.16.0
I0613 12:39:43.639795 36348 image.go:177] daemon lookup for registry.k8s.io/etcd:3.3.15-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.3.15-0
I0613 12:39:44.857154 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0613 12:39:44.999755 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/pause:3.1
I0613 12:39:45.023390 36348 cache_images.go:116] "registry.k8s.io/pause:3.1" needs transfer: "registry.k8s.io/pause:3.1" does not exist at hash "da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e" in container runtime
I0613 12:39:45.023432 36348 docker.go:316] Removing image: registry.k8s.io/pause:3.1
I0613 12:39:45.023493 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/pause:3.1
I0613 12:39:45.045916 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1
I0613 12:39:45.230468 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/coredns:1.6.2
I0613 12:39:45.252778 36348 cache_images.go:116] "registry.k8s.io/coredns:1.6.2" needs transfer: "registry.k8s.io/coredns:1.6.2" does not exist at hash "bf261d157914477ee1a5969d28ec687f3fbfc9fb5a664b22df78e57023b0e03b" in container runtime
I0613 12:39:45.252803 36348 docker.go:316] Removing image: registry.k8s.io/coredns:1.6.2
I0613 12:39:45.252860 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/coredns:1.6.2
I0613 12:39:45.276953 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.2
I0613 12:39:45.450635 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.16.0
I0613 12:39:45.473232 36348 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.16.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.16.0" does not exist at hash "301ddc62b80b16315d3c2653cf3888370394277afb3187614cfa20edc352ca0a" in container runtime
I0613 12:39:45.473263 36348 docker.go:316] Removing image: registry.k8s.io/kube-scheduler:v1.16.0
I0613 12:39:45.473321 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-scheduler:v1.16.0
I0613 12:39:45.496014 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.16.0
I0613 12:39:45.511132 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.16.0
I0613 12:39:45.533826 36348 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.16.0" needs transfer: "registry.k8s.io/kube-proxy:v1.16.0" does not exist at hash "c21b0c7400f988db4777858edd13b6d3930d62d7ccf026d2415485a52037f384" in container runtime
I0613 12:39:45.533851 36348 docker.go:316] Removing image: registry.k8s.io/kube-proxy:v1.16.0
I0613 12:39:45.533907 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-proxy:v1.16.0
I0613 12:39:45.556762 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.16.0
I0613 12:39:45.702153 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.16.0
I0613 12:39:45.725625 36348 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.16.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.16.0" does not exist at hash "b305571ca60a5a7818bda47da122683d75e8a1907475681ee8b1efbd06bff12e" in container runtime
I0613 12:39:45.725672 36348 docker.go:316] Removing image: registry.k8s.io/kube-apiserver:v1.16.0
I0613 12:39:45.725741 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-apiserver:v1.16.0
I0613 12:39:45.750069 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.16.0
I0613 12:39:46.008099 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.16.0
I0613 12:39:46.029723 36348 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.16.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.16.0" does not exist at hash "06a629a7e51cdcc81a5ed6a3e6650348312f20c954ac52ee489a023628ec9c7d" in container runtime
I0613 12:39:46.029754 36348 docker.go:316] Removing image: registry.k8s.io/kube-controller-manager:v1.16.0
I0613 12:39:46.029822 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-controller-manager:v1.16.0
I0613 12:39:46.049899 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.16.0
I0613 12:39:46.276060 36348 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/etcd:3.3.15-0
I0613 12:39:46.298166 36348 cache_images.go:116] "registry.k8s.io/etcd:3.3.15-0" needs transfer: "registry.k8s.io/etcd:3.3.15-0" does not exist at hash "b2756210eeabf84f3221da9959e9483f3919dc2aaab4cd45e7cd072fcbde27ed" in container runtime
I0613 12:39:46.298197 36348 docker.go:316] Removing image: registry.k8s.io/etcd:3.3.15-0
I0613 12:39:46.298263 36348 ssh_runner.go:195] Run: docker rmi registry.k8s.io/etcd:3.3.15-0
I0613 12:39:46.318376 36348 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.3.15-0
I0613 12:39:46.318425 36348 cache_images.go:92] LoadImages completed in 2.694948012s
W0613 12:39:46.318476 36348 out.go:239] X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1: no such file or directory
X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/15003-20351/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1: no such file or directory
I0613 12:39:46.318559 36348 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0613 12:39:46.375068 36348 cni.go:84] Creating CNI manager for ""
I0613 12:39:46.375083 36348 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0613 12:39:46.375099 36348 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0613 12:39:46.375122 36348 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-554000 NodeName:old-k8s-version-554000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0613 12:39:46.375243 36348 kubeadm.go:181] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-554000"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-554000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0613 12:39:46.375314 36348 kubeadm.go:976] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-554000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-554000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0613 12:39:46.375381 36348 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0613 12:39:46.384874 36348 binaries.go:44] Found k8s binaries, skipping transfer
I0613 12:39:46.384938 36348 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0613 12:39:46.393808 36348 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0613 12:39:46.410159 36348 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0613 12:39:46.427060 36348 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0613 12:39:46.444344 36348 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0613 12:39:46.449026 36348 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0613 12:39:46.460378 36348 certs.go:56] Setting up /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000 for IP: 192.168.67.2
I0613 12:39:46.460397 36348 certs.go:190] acquiring lock for shared ca certs: {Name:mk20811674ea367fa17992256fb23dfacc431c35 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.460575 36348 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.key
I0613 12:39:46.460632 36348 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15003-20351/.minikube/proxy-client-ca.key
I0613 12:39:46.460677 36348 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.key
I0613 12:39:46.460691 36348 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.crt with IP's: []
I0613 12:39:46.543033 36348 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.crt ...
I0613 12:39:46.543044 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.crt: {Name:mk6bd7faf5f189738ebcf9933816b46aecb99d25 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.543348 36348 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.key ...
I0613 12:39:46.543356 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/client.key: {Name:mke7f89b16386bca991e115386bea32c40a66c49 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.543555 36348 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key.c7fa3a9e
I0613 12:39:46.543568 36348 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt.c7fa3a9e with IP's: [192.168.67.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0613 12:39:46.688163 36348 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt.c7fa3a9e ...
I0613 12:39:46.688183 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt.c7fa3a9e: {Name:mka99eace4874076692c1fb514adbbcdefb826a4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.688511 36348 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key.c7fa3a9e ...
I0613 12:39:46.688521 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key.c7fa3a9e: {Name:mk9356f816c0be912da7d6566ebe44b5eb9b5e1d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.688706 36348 certs.go:337] copying /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt.c7fa3a9e -> /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt
I0613 12:39:46.688869 36348 certs.go:341] copying /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key.c7fa3a9e -> /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key
I0613 12:39:46.689023 36348 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.key
I0613 12:39:46.689036 36348 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.crt with IP's: []
I0613 12:39:46.804974 36348 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.crt ...
I0613 12:39:46.804986 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.crt: {Name:mk1dec45786b62344ec40209fb6e8abc40cfd2fb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.805275 36348 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.key ...
I0613 12:39:46.805297 36348 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.key: {Name:mkf79aaad06d222399124c56c63315b6d13ee1d5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0613 12:39:46.805717 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/20800.pem (1338 bytes)
W0613 12:39:46.805821 36348 certs.go:433] ignoring /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/20800_empty.pem, impossibly tiny 0 bytes
I0613 12:39:46.805846 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca-key.pem (1679 bytes)
I0613 12:39:46.805891 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/ca.pem (1082 bytes)
I0613 12:39:46.805932 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/cert.pem (1123 bytes)
I0613 12:39:46.805973 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/certs/key.pem (1679 bytes)
I0613 12:39:46.806056 36348 certs.go:437] found cert: /Users/jenkins/minikube-integration/15003-20351/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15003-20351/.minikube/files/etc/ssl/certs/208002.pem (1708 bytes)
I0613 12:39:46.806729 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0613 12:39:46.830731 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0613 12:39:46.853010 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0613 12:39:46.875084 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/profiles/old-k8s-version-554000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0613 12:39:46.897065 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0613 12:39:46.919385 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0613 12:39:46.941920 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0613 12:39:46.963991 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0613 12:39:46.986199 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0613 12:39:47.008241 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/certs/20800.pem --> /usr/share/ca-certificates/20800.pem (1338 bytes)
I0613 12:39:47.031230 36348 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15003-20351/.minikube/files/etc/ssl/certs/208002.pem --> /usr/share/ca-certificates/208002.pem (1708 bytes)
I0613 12:39:47.053615 36348 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0613 12:39:47.070322 36348 ssh_runner.go:195] Run: openssl version
I0613 12:39:47.076839 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0613 12:39:47.086578 36348 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0613 12:39:47.091002 36348 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Jun 13 18:43 /usr/share/ca-certificates/minikubeCA.pem
I0613 12:39:47.091044 36348 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0613 12:39:47.098042 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0613 12:39:47.108168 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/20800.pem && ln -fs /usr/share/ca-certificates/20800.pem /etc/ssl/certs/20800.pem"
I0613 12:39:47.117962 36348 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/20800.pem
I0613 12:39:47.122504 36348 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Jun 13 18:49 /usr/share/ca-certificates/20800.pem
I0613 12:39:47.122587 36348 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/20800.pem
I0613 12:39:47.129825 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/20800.pem /etc/ssl/certs/51391683.0"
I0613 12:39:47.139660 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/208002.pem && ln -fs /usr/share/ca-certificates/208002.pem /etc/ssl/certs/208002.pem"
I0613 12:39:47.149254 36348 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/208002.pem
I0613 12:39:47.154385 36348 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Jun 13 18:49 /usr/share/ca-certificates/208002.pem
I0613 12:39:47.154439 36348 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/208002.pem
I0613 12:39:47.162305 36348 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/208002.pem /etc/ssl/certs/3ec20f2e.0"
I0613 12:39:47.173440 36348 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd
I0613 12:39:47.178918 36348 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2
stdout:
stderr:
ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory
I0613 12:39:47.178966 36348 kubeadm.go:404] StartCluster: {Name:old-k8s-version-554000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-554000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0613 12:39:47.179066 36348 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0613 12:39:47.204363 36348 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0613 12:39:47.215794 36348 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0613 12:39:47.226215 36348 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0613 12:39:47.226280 36348 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0613 12:39:47.237663 36348 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0613 12:39:47.237705 36348 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0613 12:39:47.295148 36348 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0613 12:39:47.295208 36348 kubeadm.go:322] [preflight] Running pre-flight checks
I0613 12:39:47.595791 36348 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0613 12:39:47.595903 36348 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0613 12:39:47.595989 36348 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0613 12:39:47.815537 36348 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0613 12:39:47.816441 36348 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0613 12:39:47.824293 36348 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0613 12:39:47.896220 36348 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0613 12:39:47.917637 36348 out.go:204] - Generating certificates and keys ...
I0613 12:39:47.917715 36348 kubeadm.go:322] [certs] Using existing ca certificate authority
I0613 12:39:47.917793 36348 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0613 12:39:48.147497 36348 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0613 12:39:48.255669 36348 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0613 12:39:48.371435 36348 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0613 12:39:48.606324 36348 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0613 12:39:48.740373 36348 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0613 12:39:48.741068 36348 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0613 12:39:48.831498 36348 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0613 12:39:48.831646 36348 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0613 12:39:49.029205 36348 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0613 12:39:49.149463 36348 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0613 12:39:49.252660 36348 kubeadm.go:322] [certs] Generating "sa" key and public key
I0613 12:39:49.252868 36348 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0613 12:39:49.376258 36348 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0613 12:39:49.451412 36348 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0613 12:39:49.567056 36348 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0613 12:39:49.717127 36348 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0613 12:39:49.717677 36348 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0613 12:39:49.739160 36348 out.go:204] - Booting up control plane ...
I0613 12:39:49.739250 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0613 12:39:49.739337 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0613 12:39:49.739411 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0613 12:39:49.739511 36348 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0613 12:39:49.739640 36348 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0613 12:40:29.725276 36348 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0613 12:40:29.725789 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:40:29.725953 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:40:34.726680 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:40:34.726904 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:40:44.726982 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:40:44.727436 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:41:04.727890 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:41:04.728071 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:41:44.729163 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:41:44.729383 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:41:44.729402 36348 kubeadm.go:322]
I0613 12:41:44.729460 36348 kubeadm.go:322] Unfortunately, an error has occurred:
I0613 12:41:44.729519 36348 kubeadm.go:322] timed out waiting for the condition
I0613 12:41:44.729528 36348 kubeadm.go:322]
I0613 12:41:44.729574 36348 kubeadm.go:322] This error is likely caused by:
I0613 12:41:44.729642 36348 kubeadm.go:322] - The kubelet is not running
I0613 12:41:44.729813 36348 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0613 12:41:44.729828 36348 kubeadm.go:322]
I0613 12:41:44.729954 36348 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0613 12:41:44.730002 36348 kubeadm.go:322] - 'systemctl status kubelet'
I0613 12:41:44.730046 36348 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0613 12:41:44.730054 36348 kubeadm.go:322]
I0613 12:41:44.730178 36348 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0613 12:41:44.730305 36348 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0613 12:41:44.730397 36348 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0613 12:41:44.730457 36348 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0613 12:41:44.730519 36348 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0613 12:41:44.730542 36348 kubeadm.go:322] - 'docker logs CONTAINERID'
I0613 12:41:44.733820 36348 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0613 12:41:44.733896 36348 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0613 12:41:44.733999 36348 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
I0613 12:41:44.734094 36348 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0613 12:41:44.734173 36348 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0613 12:41:44.734249 36348 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0613 12:41:44.734363 36348 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-554000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0613 12:41:44.734393 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0613 12:41:45.153249 36348 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0613 12:41:45.164615 36348 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0613 12:41:45.164676 36348 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0613 12:41:45.173702 36348 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0613 12:41:45.173727 36348 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0613 12:41:45.225693 36348 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0613 12:41:45.225747 36348 kubeadm.go:322] [preflight] Running pre-flight checks
I0613 12:41:45.476723 36348 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0613 12:41:45.476821 36348 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0613 12:41:45.476915 36348 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0613 12:41:45.661298 36348 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0613 12:41:45.662256 36348 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0613 12:41:45.668983 36348 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0613 12:41:45.735752 36348 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0613 12:41:45.777899 36348 out.go:204] - Generating certificates and keys ...
I0613 12:41:45.777990 36348 kubeadm.go:322] [certs] Using existing ca certificate authority
I0613 12:41:45.778062 36348 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0613 12:41:45.778154 36348 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0613 12:41:45.778227 36348 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0613 12:41:45.778298 36348 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0613 12:41:45.778344 36348 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0613 12:41:45.778439 36348 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0613 12:41:45.778496 36348 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0613 12:41:45.778562 36348 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0613 12:41:45.778624 36348 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0613 12:41:45.778665 36348 kubeadm.go:322] [certs] Using the existing "sa" key
I0613 12:41:45.778719 36348 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0613 12:41:45.843851 36348 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0613 12:41:45.986609 36348 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0613 12:41:46.454797 36348 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0613 12:41:46.598554 36348 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0613 12:41:46.598922 36348 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0613 12:41:46.620280 36348 out.go:204] - Booting up control plane ...
I0613 12:41:46.620513 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0613 12:41:46.620688 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0613 12:41:46.620813 36348 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0613 12:41:46.620987 36348 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0613 12:41:46.621316 36348 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0613 12:42:26.607770 36348 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0613 12:42:26.608860 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:42:26.609088 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:42:31.610925 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:42:31.611183 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:42:41.612271 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:42:41.612544 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:43:01.614843 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:43:01.615069 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:43:41.615800 36348 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0613 12:43:41.615993 36348 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0613 12:43:41.616019 36348 kubeadm.go:322]
I0613 12:43:41.616076 36348 kubeadm.go:322] Unfortunately, an error has occurred:
I0613 12:43:41.616138 36348 kubeadm.go:322] timed out waiting for the condition
I0613 12:43:41.616152 36348 kubeadm.go:322]
I0613 12:43:41.616188 36348 kubeadm.go:322] This error is likely caused by:
I0613 12:43:41.616233 36348 kubeadm.go:322] - The kubelet is not running
I0613 12:43:41.616361 36348 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0613 12:43:41.616371 36348 kubeadm.go:322]
I0613 12:43:41.616520 36348 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0613 12:43:41.616561 36348 kubeadm.go:322] - 'systemctl status kubelet'
I0613 12:43:41.616596 36348 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0613 12:43:41.616605 36348 kubeadm.go:322]
I0613 12:43:41.616718 36348 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0613 12:43:41.616836 36348 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0613 12:43:41.616950 36348 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0613 12:43:41.617013 36348 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0613 12:43:41.617105 36348 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0613 12:43:41.617159 36348 kubeadm.go:322] - 'docker logs CONTAINERID'
I0613 12:43:41.620175 36348 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0613 12:43:41.620239 36348 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0613 12:43:41.620339 36348 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
I0613 12:43:41.620412 36348 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0613 12:43:41.620471 36348 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0613 12:43:41.620532 36348 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0613 12:43:41.620553 36348 kubeadm.go:406] StartCluster complete in 3m54.446543643s
I0613 12:43:41.620661 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0613 12:43:41.642283 36348 logs.go:284] 0 containers: []
W0613 12:43:41.642299 36348 logs.go:286] No container was found matching "kube-apiserver"
I0613 12:43:41.642372 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0613 12:43:41.665176 36348 logs.go:284] 0 containers: []
W0613 12:43:41.665191 36348 logs.go:286] No container was found matching "etcd"
I0613 12:43:41.665268 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0613 12:43:41.685116 36348 logs.go:284] 0 containers: []
W0613 12:43:41.685130 36348 logs.go:286] No container was found matching "coredns"
I0613 12:43:41.685206 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0613 12:43:41.714074 36348 logs.go:284] 0 containers: []
W0613 12:43:41.714088 36348 logs.go:286] No container was found matching "kube-scheduler"
I0613 12:43:41.714157 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0613 12:43:41.737492 36348 logs.go:284] 0 containers: []
W0613 12:43:41.737516 36348 logs.go:286] No container was found matching "kube-proxy"
I0613 12:43:41.737637 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0613 12:43:41.758059 36348 logs.go:284] 0 containers: []
W0613 12:43:41.758073 36348 logs.go:286] No container was found matching "kube-controller-manager"
I0613 12:43:41.758140 36348 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0613 12:43:41.780095 36348 logs.go:284] 0 containers: []
W0613 12:43:41.780108 36348 logs.go:286] No container was found matching "kindnet"
I0613 12:43:41.780115 36348 logs.go:123] Gathering logs for kubelet ...
I0613 12:43:41.780123 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0613 12:43:41.818743 36348 logs.go:123] Gathering logs for dmesg ...
I0613 12:43:41.818756 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0613 12:43:41.833042 36348 logs.go:123] Gathering logs for describe nodes ...
I0613 12:43:41.833060 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0613 12:43:41.891708 36348 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0613 12:43:41.891723 36348 logs.go:123] Gathering logs for Docker ...
I0613 12:43:41.891731 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0613 12:43:41.908742 36348 logs.go:123] Gathering logs for container status ...
I0613 12:43:41.908756 36348 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0613 12:43:41.961114 36348 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0613 12:43:41.961136 36348 out.go:239] *
*
W0613 12:43:41.961175 36348 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0613 12:43:41.961190 36348 out.go:239] *
*
W0613 12:43:41.961804 36348 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0613 12:43:42.024267 36348 out.go:177]
W0613 12:43:42.087625 36348 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.2. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0613 12:43:42.087694 36348 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0613 12:43:42.087723 36348 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0613 12:43:42.109571 36348 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-554000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-554000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-554000:
-- stdout --
[
{
"Id": "48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783",
"Created": "2023-06-13T19:39:30.478126252Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 681849,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-06-13T19:39:30.729047225Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:8b39c0c6b43e13425df6546d3707123c5158cae4cca961fab19bf263071fc26b",
"ResolvConfPath": "/var/lib/docker/containers/48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783/hostname",
"HostsPath": "/var/lib/docker/containers/48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783/hosts",
"LogPath": "/var/lib/docker/containers/48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783/48c9404a6c77e63d13b8527df2e766f7494743f6ed8ef4ef7d353a3c849b5783-json.log",
"Name": "/old-k8s-version-554000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-554000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-554000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/de878bcaa749eafa8ac71244ed4b4d7d8d82a97c174c77cf81b9fcb77569f3df-init/diff:/var/lib/docker/overlay2/74bda2c4a8f6b504659e5538348208ad822d571d6fa10595261cb1058929d560/diff",
"MergedDir": "/var/lib/docker/overlay2/de878bcaa749eafa8ac71244ed4b4d7d8d82a97c174c77cf81b9fcb77569f3df/merged",
"UpperDir": "/var/lib/docker/overlay2/de878bcaa749eafa8ac71244ed4b4d7d8d82a97c174c77cf81b9fcb77569f3df/diff",
"WorkDir": "/var/lib/docker/overlay2/de878bcaa749eafa8ac71244ed4b4d7d8d82a97c174c77cf81b9fcb77569f3df/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-554000",
"Source": "/var/lib/docker/volumes/old-k8s-version-554000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-554000",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1686006988-16632@sha256:412dc5cf58908f3565f59ed5f2b8341f53e998f8d8b54f59253c8f8f335f5a7b",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-554000",
"name.minikube.sigs.k8s.io": "old-k8s-version-554000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "ccc5d1a801b8383a39d53ac00e429cf0a4ff856cf607fb13298c236e5594fd36",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59436"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59437"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59438"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59434"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59435"
}
]
},
"SandboxKey": "/var/run/docker/netns/ccc5d1a801b8",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-554000": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"48c9404a6c77",
"old-k8s-version-554000"
],
"NetworkID": "a540b9a7399007dd984f1d1dd3d2c1d41a530fe412b07cd734dab04e1b578830",
"EndpointID": "2da931a3219cfcd17b0d7372b6b5662dc0f69e7d360759fd8c64b0cedd7cd9bc",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-554000 -n old-k8s-version-554000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-554000 -n old-k8s-version-554000: exit status 6 (357.786819ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0613 12:43:42.612632 37507 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-554000" does not appear in /Users/jenkins/minikube-integration/15003-20351/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-554000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (259.85s)