=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-919000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-919000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m10.04334901s)
-- stdout --
* [old-k8s-version-919000] minikube v1.29.0 on Darwin 13.2
- MINIKUBE_LOCATION=15909
- KUBECONFIG=/Users/jenkins/minikube-integration/15909-14738/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-14738/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-919000 in cluster old-k8s-version-919000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0223 14:52:39.431113 32430 out.go:296] Setting OutFile to fd 1 ...
I0223 14:52:39.431276 32430 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0223 14:52:39.431281 32430 out.go:309] Setting ErrFile to fd 2...
I0223 14:52:39.431285 32430 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0223 14:52:39.431393 32430 root.go:336] Updating PATH: /Users/jenkins/minikube-integration/15909-14738/.minikube/bin
I0223 14:52:39.432719 32430 out.go:303] Setting JSON to false
I0223 14:52:39.451605 32430 start.go:125] hostinfo: {"hostname":"MacOS-Agent-4.local","uptime":8533,"bootTime":1677184226,"procs":391,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.2","kernelVersion":"22.3.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"f2f27e25-cfda-5ffd-9706-e98286194e62"}
W0223 14:52:39.451734 32430 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0223 14:52:39.473576 32430 out.go:177] * [old-k8s-version-919000] minikube v1.29.0 on Darwin 13.2
I0223 14:52:39.515434 32430 notify.go:220] Checking for updates...
I0223 14:52:39.537622 32430 out.go:177] - MINIKUBE_LOCATION=15909
I0223 14:52:39.560631 32430 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15909-14738/kubeconfig
I0223 14:52:39.581424 32430 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0223 14:52:39.602621 32430 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0223 14:52:39.623530 32430 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-14738/.minikube
I0223 14:52:39.644386 32430 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0223 14:52:39.665807 32430 config.go:182] Loaded profile config "kubenet-452000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0223 14:52:39.665859 32430 driver.go:365] Setting default libvirt URI to qemu:///system
I0223 14:52:39.727929 32430 docker.go:121] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0223 14:52:39.728062 32430 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0223 14:52:39.902403 32430 info.go:266] docker info: {ID:RH4V:QTTE:6TLS:W74U:72T3:655A:HQAB:RRVU:2KDD:S3E6:3223:HKLC Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:62 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-23 22:52:39.804630495 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0223 14:52:39.944636 32430 out.go:177] * Using the docker driver based on user configuration
I0223 14:52:39.965897 32430 start.go:296] selected driver: docker
I0223 14:52:39.965921 32430 start.go:857] validating driver "docker" against <nil>
I0223 14:52:39.965946 32430 start.go:868] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0223 14:52:39.969729 32430 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0223 14:52:40.110016 32430 info.go:266] docker info: {ID:RH4V:QTTE:6TLS:W74U:72T3:655A:HQAB:RRVU:2KDD:S3E6:3223:HKLC Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:62 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-23 22:52:40.018841565 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0223 14:52:40.110146 32430 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0223 14:52:40.110325 32430 start_flags.go:919] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0223 14:52:40.132195 32430 out.go:177] * Using Docker Desktop driver with root privileges
I0223 14:52:40.153716 32430 cni.go:84] Creating CNI manager for ""
I0223 14:52:40.153755 32430 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0223 14:52:40.153768 32430 start_flags.go:319] config:
{Name:old-k8s-version-919000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-919000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0223 14:52:40.196705 32430 out.go:177] * Starting control plane node old-k8s-version-919000 in cluster old-k8s-version-919000
I0223 14:52:40.217673 32430 cache.go:120] Beginning downloading kic base image for docker with docker
I0223 14:52:40.227919 32430 out.go:177] * Pulling base image ...
I0223 14:52:40.301849 32430 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon
I0223 14:52:40.301868 32430 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 14:52:40.301956 32430 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15909-14738/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0223 14:52:40.301984 32430 cache.go:57] Caching tarball of preloaded images
I0223 14:52:40.302227 32430 preload.go:174] Found /Users/jenkins/minikube-integration/15909-14738/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0223 14:52:40.302253 32430 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0223 14:52:40.303214 32430 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/config.json ...
I0223 14:52:40.303349 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/config.json: {Name:mk1198d9bc72d8aae8620600b1c49fe68054cdad Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:40.358701 32430 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon, skipping pull
I0223 14:52:40.358719 32430 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc exists in daemon, skipping load
I0223 14:52:40.358740 32430 cache.go:193] Successfully downloaded all kic artifacts
I0223 14:52:40.358797 32430 start.go:364] acquiring machines lock for old-k8s-version-919000: {Name:mk1103874b67893bb0fc52742240655036abe57b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0223 14:52:40.358949 32430 start.go:368] acquired machines lock for "old-k8s-version-919000" in 139.581µs
I0223 14:52:40.358985 32430 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-919000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-919000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0223 14:52:40.359090 32430 start.go:125] createHost starting for "" (driver="docker")
I0223 14:52:40.380709 32430 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0223 14:52:40.381150 32430 start.go:159] libmachine.API.Create for "old-k8s-version-919000" (driver="docker")
I0223 14:52:40.381187 32430 client.go:168] LocalClient.Create starting
I0223 14:52:40.381397 32430 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem
I0223 14:52:40.381492 32430 main.go:141] libmachine: Decoding PEM data...
I0223 14:52:40.381536 32430 main.go:141] libmachine: Parsing certificate...
I0223 14:52:40.381667 32430 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/cert.pem
I0223 14:52:40.381731 32430 main.go:141] libmachine: Decoding PEM data...
I0223 14:52:40.381747 32430 main.go:141] libmachine: Parsing certificate...
I0223 14:52:40.382565 32430 cli_runner.go:164] Run: docker network inspect old-k8s-version-919000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0223 14:52:40.438524 32430 cli_runner.go:211] docker network inspect old-k8s-version-919000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0223 14:52:40.438627 32430 network_create.go:281] running [docker network inspect old-k8s-version-919000] to gather additional debugging logs...
I0223 14:52:40.438649 32430 cli_runner.go:164] Run: docker network inspect old-k8s-version-919000
W0223 14:52:40.492791 32430 cli_runner.go:211] docker network inspect old-k8s-version-919000 returned with exit code 1
I0223 14:52:40.492820 32430 network_create.go:284] error running [docker network inspect old-k8s-version-919000]: docker network inspect old-k8s-version-919000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-919000
I0223 14:52:40.492836 32430 network_create.go:286] output of [docker network inspect old-k8s-version-919000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-919000
** /stderr **
I0223 14:52:40.492938 32430 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0223 14:52:40.549524 32430 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0223 14:52:40.549865 32430 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc001401b60}
I0223 14:52:40.549879 32430 network_create.go:123] attempt to create docker network old-k8s-version-919000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 1500 ...
I0223 14:52:40.549942 32430 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-919000 old-k8s-version-919000
W0223 14:52:40.605268 32430 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-919000 old-k8s-version-919000 returned with exit code 1
W0223 14:52:40.605302 32430 network_create.go:148] failed to create docker network old-k8s-version-919000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 1500: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-919000 old-k8s-version-919000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0223 14:52:40.605316 32430 network_create.go:115] failed to create docker network old-k8s-version-919000 192.168.58.0/24, will retry: subnet is taken
I0223 14:52:40.606686 32430 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0223 14:52:40.607002 32430 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000ecb3d0}
I0223 14:52:40.607015 32430 network_create.go:123] attempt to create docker network old-k8s-version-919000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 1500 ...
I0223 14:52:40.607092 32430 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-919000 old-k8s-version-919000
I0223 14:52:40.695304 32430 network_create.go:107] docker network old-k8s-version-919000 192.168.67.0/24 created
I0223 14:52:40.695338 32430 kic.go:117] calculated static IP "192.168.67.2" for the "old-k8s-version-919000" container
I0223 14:52:40.695460 32430 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0223 14:52:40.752644 32430 cli_runner.go:164] Run: docker volume create old-k8s-version-919000 --label name.minikube.sigs.k8s.io=old-k8s-version-919000 --label created_by.minikube.sigs.k8s.io=true
I0223 14:52:40.808060 32430 oci.go:103] Successfully created a docker volume old-k8s-version-919000
I0223 14:52:40.808195 32430 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-919000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-919000 --entrypoint /usr/bin/test -v old-k8s-version-919000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -d /var/lib
I0223 14:52:41.253209 32430 oci.go:107] Successfully prepared a docker volume old-k8s-version-919000
I0223 14:52:41.253243 32430 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 14:52:41.253260 32430 kic.go:190] Starting extracting preloaded images to volume ...
I0223 14:52:41.253358 32430 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-14738/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-919000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir
I0223 14:52:46.908882 32430 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-14738/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-919000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir: (5.655294557s)
I0223 14:52:46.908905 32430 kic.go:199] duration metric: took 5.655481 seconds to extract preloaded images to volume
I0223 14:52:46.909018 32430 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0223 14:52:47.054050 32430 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-919000 --name old-k8s-version-919000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-919000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-919000 --network old-k8s-version-919000 --ip 192.168.67.2 --volume old-k8s-version-919000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc
I0223 14:52:47.402867 32430 cli_runner.go:164] Run: docker container inspect old-k8s-version-919000 --format={{.State.Running}}
I0223 14:52:47.462729 32430 cli_runner.go:164] Run: docker container inspect old-k8s-version-919000 --format={{.State.Status}}
I0223 14:52:47.522800 32430 cli_runner.go:164] Run: docker exec old-k8s-version-919000 stat /var/lib/dpkg/alternatives/iptables
I0223 14:52:47.629111 32430 oci.go:144] the created container "old-k8s-version-919000" has a running status.
I0223 14:52:47.629151 32430 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa...
I0223 14:52:47.739665 32430 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0223 14:52:47.900573 32430 cli_runner.go:164] Run: docker container inspect old-k8s-version-919000 --format={{.State.Status}}
I0223 14:52:47.970228 32430 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0223 14:52:47.970255 32430 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-919000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0223 14:52:48.082149 32430 cli_runner.go:164] Run: docker container inspect old-k8s-version-919000 --format={{.State.Status}}
I0223 14:52:48.145203 32430 machine.go:88] provisioning docker machine ...
I0223 14:52:48.145257 32430 ubuntu.go:169] provisioning hostname "old-k8s-version-919000"
I0223 14:52:48.145379 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:48.212013 32430 main.go:141] libmachine: Using SSH client type: native
I0223 14:52:48.212460 32430 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 62097 <nil> <nil>}
I0223 14:52:48.212480 32430 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-919000 && echo "old-k8s-version-919000" | sudo tee /etc/hostname
I0223 14:52:48.370569 32430 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-919000
I0223 14:52:48.370686 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:48.436912 32430 main.go:141] libmachine: Using SSH client type: native
I0223 14:52:48.437282 32430 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 62097 <nil> <nil>}
I0223 14:52:48.437295 32430 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-919000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-919000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-919000' | sudo tee -a /etc/hosts;
fi
fi
I0223 14:52:48.573405 32430 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0223 14:52:48.573440 32430 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15909-14738/.minikube CaCertPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15909-14738/.minikube}
I0223 14:52:48.573470 32430 ubuntu.go:177] setting up certificates
I0223 14:52:48.573476 32430 provision.go:83] configureAuth start
I0223 14:52:48.573551 32430 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-919000
I0223 14:52:48.637083 32430 provision.go:138] copyHostCerts
I0223 14:52:48.637187 32430 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.pem, removing ...
I0223 14:52:48.637197 32430 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.pem
I0223 14:52:48.637309 32430 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.pem (1082 bytes)
I0223 14:52:48.637500 32430 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-14738/.minikube/cert.pem, removing ...
I0223 14:52:48.637507 32430 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-14738/.minikube/cert.pem
I0223 14:52:48.637580 32430 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15909-14738/.minikube/cert.pem (1123 bytes)
I0223 14:52:48.637747 32430 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-14738/.minikube/key.pem, removing ...
I0223 14:52:48.637753 32430 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-14738/.minikube/key.pem
I0223 14:52:48.637822 32430 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15909-14738/.minikube/key.pem (1675 bytes)
I0223 14:52:48.637946 32430 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-919000 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-919000]
I0223 14:52:48.732495 32430 provision.go:172] copyRemoteCerts
I0223 14:52:48.732561 32430 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0223 14:52:48.732617 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:48.800182 32430 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:62097 SSHKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa Username:docker}
I0223 14:52:48.897431 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0223 14:52:48.923817 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0223 14:52:48.947292 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0223 14:52:48.969499 32430 provision.go:86] duration metric: configureAuth took 395.999881ms
I0223 14:52:48.969514 32430 ubuntu.go:193] setting minikube options for container-runtime
I0223 14:52:48.969683 32430 config.go:182] Loaded profile config "old-k8s-version-919000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0223 14:52:48.969744 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:49.035347 32430 main.go:141] libmachine: Using SSH client type: native
I0223 14:52:49.035787 32430 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 62097 <nil> <nil>}
I0223 14:52:49.035816 32430 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0223 14:52:49.168398 32430 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0223 14:52:49.168411 32430 ubuntu.go:71] root file system type: overlay
I0223 14:52:49.168494 32430 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0223 14:52:49.168582 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:49.235073 32430 main.go:141] libmachine: Using SSH client type: native
I0223 14:52:49.235438 32430 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 62097 <nil> <nil>}
I0223 14:52:49.235490 32430 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0223 14:52:49.378324 32430 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0223 14:52:49.378411 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:49.460875 32430 main.go:141] libmachine: Using SSH client type: native
I0223 14:52:49.461242 32430 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 62097 <nil> <nil>}
I0223 14:52:49.461258 32430 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0223 14:52:50.231373 32430 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-02-09 19:46:56.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-02-23 22:52:49.374491755 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0223 14:52:50.231397 32430 machine.go:91] provisioned docker machine in 2.086103043s
I0223 14:52:50.231404 32430 client.go:171] LocalClient.Create took 9.84992635s
I0223 14:52:50.231422 32430 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-919000" took 9.849988324s
I0223 14:52:50.231434 32430 start.go:300] post-start starting for "old-k8s-version-919000" (driver="docker")
I0223 14:52:50.231440 32430 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0223 14:52:50.231536 32430 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0223 14:52:50.231603 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:50.296889 32430 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:62097 SSHKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa Username:docker}
I0223 14:52:50.392588 32430 ssh_runner.go:195] Run: cat /etc/os-release
I0223 14:52:50.396770 32430 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0223 14:52:50.396788 32430 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0223 14:52:50.396800 32430 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0223 14:52:50.396805 32430 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0223 14:52:50.396816 32430 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-14738/.minikube/addons for local assets ...
I0223 14:52:50.396915 32430 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-14738/.minikube/files for local assets ...
I0223 14:52:50.397091 32430 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15909-14738/.minikube/files/etc/ssl/certs/152102.pem -> 152102.pem in /etc/ssl/certs
I0223 14:52:50.397315 32430 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0223 14:52:50.405426 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/files/etc/ssl/certs/152102.pem --> /etc/ssl/certs/152102.pem (1708 bytes)
I0223 14:52:50.426644 32430 start.go:303] post-start completed in 195.194102ms
I0223 14:52:50.427202 32430 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-919000
I0223 14:52:50.488702 32430 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/config.json ...
I0223 14:52:50.489153 32430 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0223 14:52:50.489208 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:50.553273 32430 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:62097 SSHKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa Username:docker}
I0223 14:52:50.643080 32430 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0223 14:52:50.647735 32430 start.go:128] duration metric: createHost completed in 10.288336174s
I0223 14:52:50.647757 32430 start.go:83] releasing machines lock for "old-k8s-version-919000", held for 10.288503047s
I0223 14:52:50.647851 32430 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-919000
I0223 14:52:50.707122 32430 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0223 14:52:50.707123 32430 ssh_runner.go:195] Run: cat /version.json
I0223 14:52:50.707221 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:50.707244 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:50.771087 32430 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:62097 SSHKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa Username:docker}
I0223 14:52:50.772513 32430 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:62097 SSHKeyPath:/Users/jenkins/minikube-integration/15909-14738/.minikube/machines/old-k8s-version-919000/id_rsa Username:docker}
I0223 14:52:51.066375 32430 ssh_runner.go:195] Run: systemctl --version
I0223 14:52:51.072080 32430 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0223 14:52:51.077490 32430 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0223 14:52:51.099222 32430 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0223 14:52:51.099293 32430 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0223 14:52:51.113674 32430 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0223 14:52:51.122186 32430 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0223 14:52:51.122206 32430 start.go:485] detecting cgroup driver to use...
I0223 14:52:51.122218 32430 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0223 14:52:51.122300 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0223 14:52:51.135930 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0223 14:52:51.144433 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0223 14:52:51.152868 32430 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0223 14:52:51.152927 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0223 14:52:51.161368 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0223 14:52:51.170281 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0223 14:52:51.178618 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0223 14:52:51.186889 32430 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0223 14:52:51.194578 32430 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0223 14:52:51.203017 32430 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0223 14:52:51.210291 32430 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0223 14:52:51.217445 32430 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0223 14:52:51.288947 32430 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0223 14:52:51.359519 32430 start.go:485] detecting cgroup driver to use...
I0223 14:52:51.359539 32430 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0223 14:52:51.359598 32430 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0223 14:52:51.370179 32430 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0223 14:52:51.370249 32430 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0223 14:52:51.380482 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0223 14:52:51.394630 32430 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0223 14:52:51.461609 32430 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0223 14:52:51.553760 32430 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0223 14:52:51.553778 32430 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0223 14:52:51.568020 32430 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0223 14:52:51.651803 32430 ssh_runner.go:195] Run: sudo systemctl restart docker
I0223 14:52:51.877800 32430 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0223 14:52:51.903791 32430 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0223 14:52:51.956159 32430 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
I0223 14:52:51.956268 32430 cli_runner.go:164] Run: docker exec -t old-k8s-version-919000 dig +short host.docker.internal
I0223 14:52:52.068331 32430 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0223 14:52:52.068439 32430 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0223 14:52:52.072634 32430 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0223 14:52:52.082701 32430 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-919000
I0223 14:52:52.140954 32430 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 14:52:52.141045 32430 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0223 14:52:52.161804 32430 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0223 14:52:52.161821 32430 docker.go:560] Images already preloaded, skipping extraction
I0223 14:52:52.161898 32430 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0223 14:52:52.182412 32430 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0223 14:52:52.182429 32430 cache_images.go:84] Images are preloaded, skipping loading
I0223 14:52:52.182524 32430 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0223 14:52:52.207940 32430 cni.go:84] Creating CNI manager for ""
I0223 14:52:52.207961 32430 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0223 14:52:52.207993 32430 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0223 14:52:52.208015 32430 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-919000 NodeName:old-k8s-version-919000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0223 14:52:52.208146 32430 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-919000"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-919000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0223 14:52:52.208234 32430 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-919000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-919000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0223 14:52:52.208301 32430 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0223 14:52:52.216117 32430 binaries.go:44] Found k8s binaries, skipping transfer
I0223 14:52:52.216182 32430 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0223 14:52:52.224062 32430 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0223 14:52:52.237096 32430 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0223 14:52:52.249964 32430 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0223 14:52:52.262695 32430 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0223 14:52:52.266486 32430 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0223 14:52:52.276954 32430 certs.go:56] Setting up /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000 for IP: 192.168.67.2
I0223 14:52:52.276972 32430 certs.go:186] acquiring lock for shared ca certs: {Name:mkd042e3451e4b14920a2306f1ed09ac35ec1a58 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.277162 32430 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.key
I0223 14:52:52.277226 32430 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15909-14738/.minikube/proxy-client-ca.key
I0223 14:52:52.277273 32430 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.key
I0223 14:52:52.277285 32430 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.crt with IP's: []
I0223 14:52:52.516820 32430 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.crt ...
I0223 14:52:52.516835 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.crt: {Name:mk4729da49f295ebed842cbb7324ce0cd3983ac7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.517118 32430 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.key ...
I0223 14:52:52.517125 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/client.key: {Name:mkc2fcc739f19cfceab502db0ed3656e8e828956 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.517330 32430 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key.c7fa3a9e
I0223 14:52:52.517345 32430 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt.c7fa3a9e with IP's: [192.168.67.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0223 14:52:52.772666 32430 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt.c7fa3a9e ...
I0223 14:52:52.772684 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt.c7fa3a9e: {Name:mk52b8d2c6a741f4be45589c804e894bac175559 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.772977 32430 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key.c7fa3a9e ...
I0223 14:52:52.772986 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key.c7fa3a9e: {Name:mk606c4d2d67cc68564379563d31a8083b34c554 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.773161 32430 certs.go:333] copying /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt.c7fa3a9e -> /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt
I0223 14:52:52.773317 32430 certs.go:337] copying /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key.c7fa3a9e -> /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key
I0223 14:52:52.773460 32430 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.key
I0223 14:52:52.773474 32430 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.crt with IP's: []
I0223 14:52:52.891922 32430 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.crt ...
I0223 14:52:52.891933 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.crt: {Name:mkca411959b8fc2747c03aec93cb7e507d2c926c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.892178 32430 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.key ...
I0223 14:52:52.892190 32430 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.key: {Name:mk7a4c783c64f1c5ccbb59f7ca256b54763f19fa Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 14:52:52.892547 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/15210.pem (1338 bytes)
W0223 14:52:52.892593 32430 certs.go:397] ignoring /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/15210_empty.pem, impossibly tiny 0 bytes
I0223 14:52:52.892628 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca-key.pem (1679 bytes)
I0223 14:52:52.892671 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/ca.pem (1082 bytes)
I0223 14:52:52.892705 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/cert.pem (1123 bytes)
I0223 14:52:52.892737 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/certs/key.pem (1675 bytes)
I0223 14:52:52.892813 32430 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-14738/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15909-14738/.minikube/files/etc/ssl/certs/152102.pem (1708 bytes)
I0223 14:52:52.893274 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0223 14:52:52.911517 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0223 14:52:52.928581 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0223 14:52:52.945802 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/profiles/old-k8s-version-919000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0223 14:52:52.963694 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0223 14:52:52.981122 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0223 14:52:52.998476 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0223 14:52:53.015672 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0223 14:52:53.033013 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/files/etc/ssl/certs/152102.pem --> /usr/share/ca-certificates/152102.pem (1708 bytes)
I0223 14:52:53.050287 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0223 14:52:53.067607 32430 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-14738/.minikube/certs/15210.pem --> /usr/share/ca-certificates/15210.pem (1338 bytes)
I0223 14:52:53.084663 32430 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0223 14:52:53.097869 32430 ssh_runner.go:195] Run: openssl version
I0223 14:52:53.103683 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/152102.pem && ln -fs /usr/share/ca-certificates/152102.pem /etc/ssl/certs/152102.pem"
I0223 14:52:53.112039 32430 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/152102.pem
I0223 14:52:53.116059 32430 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Feb 23 22:04 /usr/share/ca-certificates/152102.pem
I0223 14:52:53.116112 32430 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/152102.pem
I0223 14:52:53.121627 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/152102.pem /etc/ssl/certs/3ec20f2e.0"
I0223 14:52:53.129745 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0223 14:52:53.137859 32430 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0223 14:52:53.142106 32430 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Feb 23 21:59 /usr/share/ca-certificates/minikubeCA.pem
I0223 14:52:53.142153 32430 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0223 14:52:53.147652 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0223 14:52:53.155750 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15210.pem && ln -fs /usr/share/ca-certificates/15210.pem /etc/ssl/certs/15210.pem"
I0223 14:52:53.163809 32430 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15210.pem
I0223 14:52:53.167952 32430 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Feb 23 22:04 /usr/share/ca-certificates/15210.pem
I0223 14:52:53.167998 32430 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15210.pem
I0223 14:52:53.173810 32430 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/15210.pem /etc/ssl/certs/51391683.0"
I0223 14:52:53.181719 32430 kubeadm.go:401] StartCluster: {Name:old-k8s-version-919000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-919000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0223 14:52:53.181838 32430 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0223 14:52:53.201678 32430 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0223 14:52:53.209702 32430 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0223 14:52:53.217222 32430 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0223 14:52:53.217282 32430 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0223 14:52:53.225035 32430 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0223 14:52:53.225069 32430 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0223 14:52:53.272936 32430 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0223 14:52:53.272980 32430 kubeadm.go:322] [preflight] Running pre-flight checks
I0223 14:52:53.441002 32430 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0223 14:52:53.441091 32430 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0223 14:52:53.441165 32430 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0223 14:52:53.594386 32430 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0223 14:52:53.595084 32430 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0223 14:52:53.601229 32430 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0223 14:52:53.669660 32430 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0223 14:52:53.711967 32430 out.go:204] - Generating certificates and keys ...
I0223 14:52:53.712101 32430 kubeadm.go:322] [certs] Using existing ca certificate authority
I0223 14:52:53.712188 32430 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0223 14:52:53.753918 32430 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0223 14:52:53.840933 32430 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0223 14:52:53.897455 32430 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0223 14:52:54.059363 32430 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0223 14:52:54.131936 32430 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0223 14:52:54.132053 32430 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0223 14:52:54.283583 32430 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0223 14:52:54.283701 32430 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0223 14:52:54.370331 32430 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0223 14:52:54.480788 32430 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0223 14:52:54.515290 32430 kubeadm.go:322] [certs] Generating "sa" key and public key
I0223 14:52:54.515364 32430 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0223 14:52:54.578658 32430 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0223 14:52:54.899273 32430 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0223 14:52:54.943311 32430 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0223 14:52:55.023741 32430 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0223 14:52:55.024217 32430 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0223 14:52:55.045930 32430 out.go:204] - Booting up control plane ...
I0223 14:52:55.046121 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0223 14:52:55.046241 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0223 14:52:55.046365 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0223 14:52:55.046470 32430 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0223 14:52:55.046722 32430 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0223 14:53:35.033928 32430 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0223 14:53:35.034459 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:53:35.034676 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:53:40.035363 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:53:40.035578 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:53:50.036459 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:53:50.036678 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:54:10.037838 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:54:10.038057 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:54:50.040348 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:54:50.040578 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:54:50.040590 32430 kubeadm.go:322]
I0223 14:54:50.040629 32430 kubeadm.go:322] Unfortunately, an error has occurred:
I0223 14:54:50.040681 32430 kubeadm.go:322] timed out waiting for the condition
I0223 14:54:50.040692 32430 kubeadm.go:322]
I0223 14:54:50.040730 32430 kubeadm.go:322] This error is likely caused by:
I0223 14:54:50.040763 32430 kubeadm.go:322] - The kubelet is not running
I0223 14:54:50.040917 32430 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0223 14:54:50.040932 32430 kubeadm.go:322]
I0223 14:54:50.041095 32430 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0223 14:54:50.041139 32430 kubeadm.go:322] - 'systemctl status kubelet'
I0223 14:54:50.041172 32430 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0223 14:54:50.041179 32430 kubeadm.go:322]
I0223 14:54:50.041311 32430 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0223 14:54:50.041417 32430 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0223 14:54:50.041515 32430 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0223 14:54:50.041587 32430 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0223 14:54:50.041692 32430 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0223 14:54:50.041733 32430 kubeadm.go:322] - 'docker logs CONTAINERID'
I0223 14:54:50.044081 32430 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0223 14:54:50.044149 32430 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0223 14:54:50.044255 32430 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0223 14:54:50.044341 32430 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0223 14:54:50.044416 32430 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0223 14:54:50.044479 32430 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0223 14:54:50.044632 32430 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-919000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0223 14:54:50.044663 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0223 14:54:50.476780 32430 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0223 14:54:50.486504 32430 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0223 14:54:50.486570 32430 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0223 14:54:50.494012 32430 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0223 14:54:50.494034 32430 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0223 14:54:50.540999 32430 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0223 14:54:50.541040 32430 kubeadm.go:322] [preflight] Running pre-flight checks
I0223 14:54:50.703953 32430 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0223 14:54:50.704061 32430 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0223 14:54:50.704141 32430 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0223 14:54:50.856189 32430 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0223 14:54:50.856918 32430 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0223 14:54:50.863467 32430 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0223 14:54:50.935375 32430 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0223 14:54:50.977798 32430 out.go:204] - Generating certificates and keys ...
I0223 14:54:50.977893 32430 kubeadm.go:322] [certs] Using existing ca certificate authority
I0223 14:54:50.977975 32430 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0223 14:54:50.978070 32430 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0223 14:54:50.978122 32430 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0223 14:54:50.978178 32430 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0223 14:54:50.978239 32430 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0223 14:54:50.978305 32430 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0223 14:54:50.978364 32430 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0223 14:54:50.978426 32430 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0223 14:54:50.978510 32430 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0223 14:54:50.978555 32430 kubeadm.go:322] [certs] Using the existing "sa" key
I0223 14:54:50.978610 32430 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0223 14:54:51.354698 32430 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0223 14:54:51.549943 32430 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0223 14:54:51.770655 32430 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0223 14:54:51.849426 32430 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0223 14:54:51.849997 32430 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0223 14:54:51.871383 32430 out.go:204] - Booting up control plane ...
I0223 14:54:51.871505 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0223 14:54:51.871583 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0223 14:54:51.871646 32430 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0223 14:54:51.871734 32430 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0223 14:54:51.871919 32430 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0223 14:55:31.859990 32430 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0223 14:55:31.861077 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:55:31.861281 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:55:36.862645 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:55:36.862867 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:55:46.863876 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:55:46.864112 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:56:06.866414 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:56:06.866651 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:56:46.868566 32430 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 14:56:46.868738 32430 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 14:56:46.868749 32430 kubeadm.go:322]
I0223 14:56:46.868782 32430 kubeadm.go:322] Unfortunately, an error has occurred:
I0223 14:56:46.868830 32430 kubeadm.go:322] timed out waiting for the condition
I0223 14:56:46.868851 32430 kubeadm.go:322]
I0223 14:56:46.868921 32430 kubeadm.go:322] This error is likely caused by:
I0223 14:56:46.868990 32430 kubeadm.go:322] - The kubelet is not running
I0223 14:56:46.869153 32430 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0223 14:56:46.869172 32430 kubeadm.go:322]
I0223 14:56:46.869272 32430 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0223 14:56:46.869299 32430 kubeadm.go:322] - 'systemctl status kubelet'
I0223 14:56:46.869324 32430 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0223 14:56:46.869330 32430 kubeadm.go:322]
I0223 14:56:46.869442 32430 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0223 14:56:46.869513 32430 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0223 14:56:46.869588 32430 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0223 14:56:46.869642 32430 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0223 14:56:46.869692 32430 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0223 14:56:46.869714 32430 kubeadm.go:322] - 'docker logs CONTAINERID'
I0223 14:56:46.872137 32430 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0223 14:56:46.872208 32430 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0223 14:56:46.872317 32430 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0223 14:56:46.872407 32430 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0223 14:56:46.872470 32430 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0223 14:56:46.872530 32430 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0223 14:56:46.872559 32430 kubeadm.go:403] StartCluster complete in 3m53.684090344s
I0223 14:56:46.872650 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0223 14:56:46.893196 32430 logs.go:277] 0 containers: []
W0223 14:56:46.893211 32430 logs.go:279] No container was found matching "kube-apiserver"
I0223 14:56:46.893286 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0223 14:56:46.912551 32430 logs.go:277] 0 containers: []
W0223 14:56:46.912565 32430 logs.go:279] No container was found matching "etcd"
I0223 14:56:46.912635 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0223 14:56:46.933113 32430 logs.go:277] 0 containers: []
W0223 14:56:46.933126 32430 logs.go:279] No container was found matching "coredns"
I0223 14:56:46.933197 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0223 14:56:46.951895 32430 logs.go:277] 0 containers: []
W0223 14:56:46.951911 32430 logs.go:279] No container was found matching "kube-scheduler"
I0223 14:56:46.951988 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0223 14:56:46.973476 32430 logs.go:277] 0 containers: []
W0223 14:56:46.973490 32430 logs.go:279] No container was found matching "kube-proxy"
I0223 14:56:46.973560 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0223 14:56:47.000838 32430 logs.go:277] 0 containers: []
W0223 14:56:47.000858 32430 logs.go:279] No container was found matching "kube-controller-manager"
I0223 14:56:47.000951 32430 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0223 14:56:47.020520 32430 logs.go:277] 0 containers: []
W0223 14:56:47.020535 32430 logs.go:279] No container was found matching "kindnet"
I0223 14:56:47.020543 32430 logs.go:123] Gathering logs for kubelet ...
I0223 14:56:47.020551 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0223 14:56:47.058017 32430 logs.go:123] Gathering logs for dmesg ...
I0223 14:56:47.058040 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0223 14:56:47.070651 32430 logs.go:123] Gathering logs for describe nodes ...
I0223 14:56:47.070665 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0223 14:56:47.123431 32430 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0223 14:56:47.123442 32430 logs.go:123] Gathering logs for Docker ...
I0223 14:56:47.123449 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0223 14:56:47.147608 32430 logs.go:123] Gathering logs for container status ...
I0223 14:56:47.147622 32430 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0223 14:56:49.194830 32430 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.047136646s)
W0223 14:56:49.194992 32430 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 14:56:49.195009 32430 out.go:239] *
*
W0223 14:56:49.195175 32430 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 14:56:49.195190 32430 out.go:239] *
*
W0223 14:56:49.195811 32430 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0223 14:56:49.258520 32430 out.go:177]
W0223 14:56:49.300332 32430 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 14:56:49.300408 32430 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0223 14:56:49.300452 32430 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0223 14:56:49.358626 32430 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-919000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-919000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-919000:
-- stdout --
[
{
"Id": "5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f",
"Created": "2023-02-23T22:52:47.108009889Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 274292,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-02-23T22:52:47.393667172Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:b74f629d1852fc20b6085123d98944654faddf1d7e642b41aa2866d7a48081ea",
"ResolvConfPath": "/var/lib/docker/containers/5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f/hostname",
"HostsPath": "/var/lib/docker/containers/5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f/hosts",
"LogPath": "/var/lib/docker/containers/5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f/5b30451d45705a9a685c21947c3a0ff35451f28d40836b4bc4eca950a8b5ea6f-json.log",
"Name": "/old-k8s-version-919000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-919000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-919000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/12a758cbacc32286cfb697e84377659c014b379508fef3d7d03c0d5247c9343f-init/diff:/var/lib/docker/overlay2/312af7914f267135654023cac986639fda26bce0e9e16676c1ee839dedb36ea3/diff:/var/lib/docker/overlay2/9f5e778ea554e91a930e169d54cc3039a0f410153e0eb7fd2e44371431c5239c/diff:/var/lib/docker/overlay2/21fd88361fee5b30bab54c1a2fb3661a9258260808d03a0aa5e76d695c13e9fa/diff:/var/lib/docker/overlay2/d1a70ff42b514a48ede228bfd667a1ff44276a97ca8f8972c361fbe666dbf5af/diff:/var/lib/docker/overlay2/0b3e33b93dd83274708c0ed2f844269da0eaf9b93ced47324281f889f623961f/diff:/var/lib/docker/overlay2/41ba4ebf100466946a1c040dfafdebcd1a2c3735e7fae36f117a310a88d53f27/diff:/var/lib/docker/overlay2/61da3a41b7f242cdcb824df3019a74f4cce296b58f5eb98a12aafe0f881b0b28/diff:/var/lib/docker/overlay2/1bf8b92719375a9d8f097f598013684a7349d25f3ec4b2f39c33a05d4ac38e63/diff:/var/lib/docker/overlay2/6e25221474c86778a56dad511c236c16b7f32f46f432667d5734c1c823a29c04/diff:/var/lib/docker/overlay2/516ea8
fc57230e6987a437167604d02d4c86c90cc43e30c725ebb58b328c5b28/diff:/var/lib/docker/overlay2/773735ff5815c46111f85a6a2ed29eaba38131060daeaf31fcc6d190d54c8ad0/diff:/var/lib/docker/overlay2/54f6eaef84eb22a9bd4375e213ff3f1af4d87174a0636cd705161eb9f592e76a/diff:/var/lib/docker/overlay2/c5903c40eadd84761d888193a77e1732b778ef4a0f7c591242ddd1452659e9c5/diff:/var/lib/docker/overlay2/efe55213e0610967c4943095e3d2ddc820e6be3e9832f18c669f704ba5bfb804/diff:/var/lib/docker/overlay2/dd9ef0a255fcef6df1825ec2d2f78249bdd4d29ff9b169e2bac4ec68e17ea0b5/diff:/var/lib/docker/overlay2/a88591bbe843d595c945e5ddc61dc438e66750a9f27de8cecb25a581f644f63d/diff:/var/lib/docker/overlay2/5b7a9b283ffcce0a068b6d113f8160ebffa0023496e720c09b2230405cd98660/diff:/var/lib/docker/overlay2/ba1cd99628fbd2ee5537eb57211209b402707fd4927ab6f487db64a080b2bb40/diff:/var/lib/docker/overlay2/77e297c6446310bb550315eda2e71d0ed3596dcf59cf5f929ed16415a6e839e7/diff:/var/lib/docker/overlay2/b72a642a10b9b221f8dab95965c8d7ebf61439db1817d2a7e55e3351fb3bfa79/diff:/var/lib/d
ocker/overlay2/2c85849636b2636c39c1165674634052c165bf1671737807f9f84af9cdaec710/diff:/var/lib/docker/overlay2/d481e2df4e2fbb51c3c6548fe0e2d75c3bbc6867daeaeac559fea32b0969109d/diff:/var/lib/docker/overlay2/a4ba08d7c7be1aee5f1f8ab163c91e56cc270b23926e8e8f2d6d7baee1c4cd79/diff:/var/lib/docker/overlay2/1fc8aefb80213c58eee3e457fad1ed5e0860e5c7101a8c94babf2676372d8d40/diff:/var/lib/docker/overlay2/8156590a8e10d518427298740db8a2645d4864ce4cdab44568080a1bbec209ae/diff:/var/lib/docker/overlay2/de8e7a927a81ab8b0dca0aa9ad11fb89bc2e11a56bb179b2a2a9a16246ab957d/diff:/var/lib/docker/overlay2/b1a2174e26ac2948f2a988c58c45115f230d1168b148e07573537d88cd485d27/diff:/var/lib/docker/overlay2/99eb504e3cdd219c35b20f48bd3980b389a181a64d2061645d77daee9a632a1f/diff:/var/lib/docker/overlay2/f00c0c9d98f4688c7caa116c3bef509c2aeb87bc2be717c3b4dd213a9aa6e931/diff:/var/lib/docker/overlay2/3ccdd6f5db6e7677b32d1118b2389939576cec9399a2074953bde1f44d0ffc8a/diff:/var/lib/docker/overlay2/4c71c056a816d63d030c0fff4784f0102ebcef0ab5a658ffcbe0712ec24
a9613/diff:/var/lib/docker/overlay2/3f9f8c3d456e713700ebe7d9ce7bd0ccade1486538efc09ba938942358692d6b/diff:/var/lib/docker/overlay2/6493814c93da91c97a90a193105168493b20183da8ab0a899ea52d4e893b2c49/diff:/var/lib/docker/overlay2/ad9631f623b7b3422f0937ca422d90ee0fdec23f7e5f098ec6b4997b7f779fca/diff:/var/lib/docker/overlay2/c8c5afb62a7fd536950c0205b19e9ff902be1d0392649f2bd1fcd0c8c4bf964c/diff:/var/lib/docker/overlay2/50d49e0f668e585ab4a5eebae984f585c76a14adba7817457c17a6154185262b/diff:/var/lib/docker/overlay2/5d37263f7458b15a195a8fefcae668e9bb7464e180a3c490081f228be8dbc2e6/diff:/var/lib/docker/overlay2/e82d2914dc1ce857d9e4246cfe1f5fa67768dedcf273e555191da326b0b83966/diff:/var/lib/docker/overlay2/4b3559760284dc821c75387fbf41238bdcfa44c7949d784247228e1d190e8547/diff:/var/lib/docker/overlay2/3fd6c3231524b82c531a887996ca0c4ffd24fa733444aab8fbdbf802e09e49c3/diff:/var/lib/docker/overlay2/f79c36358a76fa00014ba7ec5a0c44b160ae24ed2130967de29343cc513cb2d0/diff:/var/lib/docker/overlay2/0628686e980f429d66d25561d57e7c1cbe5405
52c70cef7d15955c6c1ad1a369/diff",
"MergedDir": "/var/lib/docker/overlay2/12a758cbacc32286cfb697e84377659c014b379508fef3d7d03c0d5247c9343f/merged",
"UpperDir": "/var/lib/docker/overlay2/12a758cbacc32286cfb697e84377659c014b379508fef3d7d03c0d5247c9343f/diff",
"WorkDir": "/var/lib/docker/overlay2/12a758cbacc32286cfb697e84377659c014b379508fef3d7d03c0d5247c9343f/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-919000",
"Source": "/var/lib/docker/volumes/old-k8s-version-919000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-919000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-919000",
"name.minikube.sigs.k8s.io": "old-k8s-version-919000",
"org.opencontainers.image.ref.name": "ubuntu",
"org.opencontainers.image.version": "20.04",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "a3aa73756f04f765ad6387b630175a269d03136baccdaf5a3a9fdc4f1198b973",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "62097"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "62098"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "62094"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "62095"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "62096"
}
]
},
"SandboxKey": "/var/run/docker/netns/a3aa73756f04",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-919000": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"5b30451d4570",
"old-k8s-version-919000"
],
"NetworkID": "c7154bbdfe1ae896999b2fd2c462dec29ff61281e64aa32aac9e788f781af78c",
"EndpointID": "ebc723f5d2c72d46fce5dc0c853fb943a03ba08eeae02776867e996748135d09",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-919000 -n old-k8s-version-919000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-919000 -n old-k8s-version-919000: exit status 6 (395.103971ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0223 14:56:49.914090 33597 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-919000" does not appear in /Users/jenkins/minikube-integration/15909-14738/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-919000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (250.53s)