=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-977000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-977000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m11.70750917s)
-- stdout --
* [old-k8s-version-977000] minikube v1.29.0 on Darwin 13.2
- MINIKUBE_LOCATION=15909
- KUBECONFIG=/Users/jenkins/minikube-integration/15909-24428/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-24428/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-977000 in cluster old-k8s-version-977000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0223 17:35:05.111638 42515 out.go:296] Setting OutFile to fd 1 ...
I0223 17:35:05.111864 42515 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0223 17:35:05.111869 42515 out.go:309] Setting ErrFile to fd 2...
I0223 17:35:05.111873 42515 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0223 17:35:05.111988 42515 root.go:336] Updating PATH: /Users/jenkins/minikube-integration/15909-24428/.minikube/bin
I0223 17:35:05.113410 42515 out.go:303] Setting JSON to false
I0223 17:35:05.132765 42515 start.go:125] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":9280,"bootTime":1677193225,"procs":436,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.2","kernelVersion":"22.3.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W0223 17:35:05.132914 42515 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0223 17:35:05.154991 42515 out.go:177] * [old-k8s-version-977000] minikube v1.29.0 on Darwin 13.2
I0223 17:35:05.197058 42515 notify.go:220] Checking for updates...
I0223 17:35:05.218165 42515 out.go:177] - MINIKUBE_LOCATION=15909
I0223 17:35:05.260953 42515 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15909-24428/kubeconfig
I0223 17:35:05.303010 42515 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0223 17:35:05.344920 42515 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0223 17:35:05.387074 42515 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-24428/.minikube
I0223 17:35:05.429164 42515 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0223 17:35:05.450953 42515 config.go:182] Loaded profile config "bridge-152000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0223 17:35:05.451066 42515 driver.go:365] Setting default libvirt URI to qemu:///system
I0223 17:35:05.514639 42515 docker.go:121] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0223 17:35:05.514765 42515 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0223 17:35:05.664419 42515 info.go:266] docker info: {ID:SWKV:KMRO:OUIS:YAN3:ZG2Q:7LAB:6DZ6:VZUC:VVW3:VUUP:WZOT:LF2A Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:63 OomKillDisable:false NGoroutines:57 SystemTime:2023-02-24 01:35:05.567273022 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0223 17:35:05.686187 42515 out.go:177] * Using the docker driver based on user configuration
I0223 17:35:05.707076 42515 start.go:296] selected driver: docker
I0223 17:35:05.707100 42515 start.go:857] validating driver "docker" against <nil>
I0223 17:35:05.707117 42515 start.go:868] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0223 17:35:05.711110 42515 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0223 17:35:05.857921 42515 info.go:266] docker info: {ID:SWKV:KMRO:OUIS:YAN3:ZG2Q:7LAB:6DZ6:VZUC:VVW3:VUUP:WZOT:LF2A Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:63 OomKillDisable:false NGoroutines:57 SystemTime:2023-02-24 01:35:05.762838571 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0223 17:35:05.858043 42515 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0223 17:35:05.858209 42515 start_flags.go:919] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0223 17:35:05.879761 42515 out.go:177] * Using Docker Desktop driver with root privileges
I0223 17:35:05.900638 42515 cni.go:84] Creating CNI manager for ""
I0223 17:35:05.900771 42515 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0223 17:35:05.900791 42515 start_flags.go:319] config:
{Name:old-k8s-version-977000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-977000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0223 17:35:05.943675 42515 out.go:177] * Starting control plane node old-k8s-version-977000 in cluster old-k8s-version-977000
I0223 17:35:05.964764 42515 cache.go:120] Beginning downloading kic base image for docker with docker
I0223 17:35:05.985749 42515 out.go:177] * Pulling base image ...
I0223 17:35:06.027712 42515 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon
I0223 17:35:06.027718 42515 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 17:35:06.027815 42515 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15909-24428/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0223 17:35:06.027839 42515 cache.go:57] Caching tarball of preloaded images
I0223 17:35:06.028124 42515 preload.go:174] Found /Users/jenkins/minikube-integration/15909-24428/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0223 17:35:06.028143 42515 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0223 17:35:06.029277 42515 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/config.json ...
I0223 17:35:06.029431 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/config.json: {Name:mke69de7ce09d338a385bae20c7027b755cb8332 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:06.085560 42515 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon, skipping pull
I0223 17:35:06.085580 42515 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc exists in daemon, skipping load
I0223 17:35:06.085600 42515 cache.go:193] Successfully downloaded all kic artifacts
I0223 17:35:06.085639 42515 start.go:364] acquiring machines lock for old-k8s-version-977000: {Name:mk29826c7430a5f84af8ee3c20735d7dd9caf7e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0223 17:35:06.085797 42515 start.go:368] acquired machines lock for "old-k8s-version-977000" in 146.084µs
I0223 17:35:06.085830 42515 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-977000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-977000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0223 17:35:06.085910 42515 start.go:125] createHost starting for "" (driver="docker")
I0223 17:35:06.144580 42515 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0223 17:35:06.144923 42515 start.go:159] libmachine.API.Create for "old-k8s-version-977000" (driver="docker")
I0223 17:35:06.144998 42515 client.go:168] LocalClient.Create starting
I0223 17:35:06.145172 42515 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem
I0223 17:35:06.145252 42515 main.go:141] libmachine: Decoding PEM data...
I0223 17:35:06.145284 42515 main.go:141] libmachine: Parsing certificate...
I0223 17:35:06.145410 42515 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/cert.pem
I0223 17:35:06.145477 42515 main.go:141] libmachine: Decoding PEM data...
I0223 17:35:06.145496 42515 main.go:141] libmachine: Parsing certificate...
I0223 17:35:06.146222 42515 cli_runner.go:164] Run: docker network inspect old-k8s-version-977000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0223 17:35:06.204335 42515 cli_runner.go:211] docker network inspect old-k8s-version-977000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0223 17:35:06.204442 42515 network_create.go:281] running [docker network inspect old-k8s-version-977000] to gather additional debugging logs...
I0223 17:35:06.204460 42515 cli_runner.go:164] Run: docker network inspect old-k8s-version-977000
W0223 17:35:06.264019 42515 cli_runner.go:211] docker network inspect old-k8s-version-977000 returned with exit code 1
I0223 17:35:06.264045 42515 network_create.go:284] error running [docker network inspect old-k8s-version-977000]: docker network inspect old-k8s-version-977000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-977000
I0223 17:35:06.264056 42515 network_create.go:286] output of [docker network inspect old-k8s-version-977000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-977000
** /stderr **
I0223 17:35:06.264159 42515 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0223 17:35:06.325263 42515 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0223 17:35:06.325633 42515 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0006df080}
I0223 17:35:06.325647 42515 network_create.go:123] attempt to create docker network old-k8s-version-977000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 1500 ...
I0223 17:35:06.325716 42515 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000
W0223 17:35:06.384171 42515 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000 returned with exit code 1
W0223 17:35:06.384202 42515 network_create.go:148] failed to create docker network old-k8s-version-977000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 1500: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0223 17:35:06.384239 42515 network_create.go:115] failed to create docker network old-k8s-version-977000 192.168.58.0/24, will retry: subnet is taken
I0223 17:35:06.385582 42515 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0223 17:35:06.385965 42515 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0010e6570}
I0223 17:35:06.385984 42515 network_create.go:123] attempt to create docker network old-k8s-version-977000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 1500 ...
I0223 17:35:06.386054 42515 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000
W0223 17:35:06.443841 42515 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000 returned with exit code 1
W0223 17:35:06.443875 42515 network_create.go:148] failed to create docker network old-k8s-version-977000 192.168.67.0/24 with gateway 192.168.67.1 and mtu of 1500: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0223 17:35:06.443890 42515 network_create.go:115] failed to create docker network old-k8s-version-977000 192.168.67.0/24, will retry: subnet is taken
I0223 17:35:06.445253 42515 network.go:212] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0223 17:35:06.445580 42515 network.go:209] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00059ec80}
I0223 17:35:06.445593 42515 network_create.go:123] attempt to create docker network old-k8s-version-977000 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0223 17:35:06.445666 42515 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-977000 old-k8s-version-977000
I0223 17:35:06.563613 42515 network_create.go:107] docker network old-k8s-version-977000 192.168.76.0/24 created
I0223 17:35:06.563656 42515 kic.go:117] calculated static IP "192.168.76.2" for the "old-k8s-version-977000" container
I0223 17:35:06.563803 42515 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0223 17:35:06.635036 42515 cli_runner.go:164] Run: docker volume create old-k8s-version-977000 --label name.minikube.sigs.k8s.io=old-k8s-version-977000 --label created_by.minikube.sigs.k8s.io=true
I0223 17:35:06.695768 42515 oci.go:103] Successfully created a docker volume old-k8s-version-977000
I0223 17:35:06.695883 42515 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-977000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-977000 --entrypoint /usr/bin/test -v old-k8s-version-977000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -d /var/lib
I0223 17:35:07.231394 42515 oci.go:107] Successfully prepared a docker volume old-k8s-version-977000
I0223 17:35:07.231430 42515 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 17:35:07.231444 42515 kic.go:190] Starting extracting preloaded images to volume ...
I0223 17:35:07.231562 42515 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-24428/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-977000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir
I0223 17:35:14.156474 42515 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-24428/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-977000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir: (6.92480532s)
I0223 17:35:14.156495 42515 kic.go:199] duration metric: took 6.925019 seconds to extract preloaded images to volume
I0223 17:35:14.156623 42515 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0223 17:35:14.314443 42515 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-977000 --name old-k8s-version-977000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-977000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-977000 --network old-k8s-version-977000 --ip 192.168.76.2 --volume old-k8s-version-977000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc
I0223 17:35:14.720287 42515 cli_runner.go:164] Run: docker container inspect old-k8s-version-977000 --format={{.State.Running}}
I0223 17:35:14.789136 42515 cli_runner.go:164] Run: docker container inspect old-k8s-version-977000 --format={{.State.Status}}
I0223 17:35:14.860525 42515 cli_runner.go:164] Run: docker exec old-k8s-version-977000 stat /var/lib/dpkg/alternatives/iptables
I0223 17:35:14.980411 42515 oci.go:144] the created container "old-k8s-version-977000" has a running status.
I0223 17:35:14.980451 42515 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa...
I0223 17:35:15.135838 42515 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0223 17:35:15.241261 42515 cli_runner.go:164] Run: docker container inspect old-k8s-version-977000 --format={{.State.Status}}
I0223 17:35:15.302006 42515 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0223 17:35:15.302026 42515 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-977000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0223 17:35:15.408013 42515 cli_runner.go:164] Run: docker container inspect old-k8s-version-977000 --format={{.State.Status}}
I0223 17:35:15.468400 42515 machine.go:88] provisioning docker machine ...
I0223 17:35:15.468443 42515 ubuntu.go:169] provisioning hostname "old-k8s-version-977000"
I0223 17:35:15.468561 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:15.534678 42515 main.go:141] libmachine: Using SSH client type: native
I0223 17:35:15.535105 42515 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61624 <nil> <nil>}
I0223 17:35:15.535119 42515 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-977000 && echo "old-k8s-version-977000" | sudo tee /etc/hostname
I0223 17:35:15.677217 42515 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-977000
I0223 17:35:15.677331 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:15.742846 42515 main.go:141] libmachine: Using SSH client type: native
I0223 17:35:15.743230 42515 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61624 <nil> <nil>}
I0223 17:35:15.743242 42515 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-977000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-977000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-977000' | sudo tee -a /etc/hosts;
fi
fi
I0223 17:35:15.877806 42515 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0223 17:35:15.877835 42515 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15909-24428/.minikube CaCertPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15909-24428/.minikube}
I0223 17:35:15.877851 42515 ubuntu.go:177] setting up certificates
I0223 17:35:15.877861 42515 provision.go:83] configureAuth start
I0223 17:35:15.877958 42515 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-977000
I0223 17:35:15.939838 42515 provision.go:138] copyHostCerts
I0223 17:35:15.939944 42515 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.pem, removing ...
I0223 17:35:15.939954 42515 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.pem
I0223 17:35:15.940053 42515 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.pem (1082 bytes)
I0223 17:35:15.940247 42515 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-24428/.minikube/cert.pem, removing ...
I0223 17:35:15.940253 42515 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-24428/.minikube/cert.pem
I0223 17:35:15.940332 42515 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15909-24428/.minikube/cert.pem (1123 bytes)
I0223 17:35:15.940507 42515 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-24428/.minikube/key.pem, removing ...
I0223 17:35:15.940513 42515 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-24428/.minikube/key.pem
I0223 17:35:15.940570 42515 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15909-24428/.minikube/key.pem (1679 bytes)
I0223 17:35:15.940698 42515 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-977000 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-977000]
I0223 17:35:16.089397 42515 provision.go:172] copyRemoteCerts
I0223 17:35:16.089496 42515 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0223 17:35:16.089551 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:16.146611 42515 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61624 SSHKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa Username:docker}
I0223 17:35:16.243942 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0223 17:35:16.264325 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0223 17:35:16.285160 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0223 17:35:16.304828 42515 provision.go:86] duration metric: configureAuth took 426.95167ms
I0223 17:35:16.304843 42515 ubuntu.go:193] setting minikube options for container-runtime
I0223 17:35:16.304996 42515 config.go:182] Loaded profile config "old-k8s-version-977000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0223 17:35:16.305065 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:16.363721 42515 main.go:141] libmachine: Using SSH client type: native
I0223 17:35:16.364082 42515 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61624 <nil> <nil>}
I0223 17:35:16.364098 42515 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0223 17:35:16.497590 42515 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0223 17:35:16.497619 42515 ubuntu.go:71] root file system type: overlay
I0223 17:35:16.497828 42515 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0223 17:35:16.497905 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:16.563338 42515 main.go:141] libmachine: Using SSH client type: native
I0223 17:35:16.563697 42515 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61624 <nil> <nil>}
I0223 17:35:16.563750 42515 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0223 17:35:16.705680 42515 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0223 17:35:16.705770 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:16.762270 42515 main.go:141] libmachine: Using SSH client type: native
I0223 17:35:16.762621 42515 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61624 <nil> <nil>}
I0223 17:35:16.762641 42515 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0223 17:35:17.381606 42515 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-02-09 19:46:56.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-02-24 01:35:16.704089528 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0223 17:35:17.381624 42515 machine.go:91] provisioned docker machine in 1.913195128s
I0223 17:35:17.381629 42515 client.go:171] LocalClient.Create took 11.236572025s
I0223 17:35:17.381645 42515 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-977000" took 11.236675601s
I0223 17:35:17.381654 42515 start.go:300] post-start starting for "old-k8s-version-977000" (driver="docker")
I0223 17:35:17.381658 42515 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0223 17:35:17.381725 42515 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0223 17:35:17.381784 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:17.440265 42515 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61624 SSHKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa Username:docker}
I0223 17:35:17.533313 42515 ssh_runner.go:195] Run: cat /etc/os-release
I0223 17:35:17.537119 42515 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0223 17:35:17.537138 42515 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0223 17:35:17.537146 42515 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0223 17:35:17.537150 42515 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0223 17:35:17.537162 42515 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-24428/.minikube/addons for local assets ...
I0223 17:35:17.537253 42515 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-24428/.minikube/files for local assets ...
I0223 17:35:17.537413 42515 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15909-24428/.minikube/files/etc/ssl/certs/248852.pem -> 248852.pem in /etc/ssl/certs
I0223 17:35:17.537576 42515 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0223 17:35:17.545280 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/files/etc/ssl/certs/248852.pem --> /etc/ssl/certs/248852.pem (1708 bytes)
I0223 17:35:17.562825 42515 start.go:303] post-start completed in 181.154814ms
I0223 17:35:17.563360 42515 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-977000
I0223 17:35:17.620215 42515 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/config.json ...
I0223 17:35:17.620629 42515 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0223 17:35:17.620694 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:17.676411 42515 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61624 SSHKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa Username:docker}
I0223 17:35:17.766403 42515 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0223 17:35:17.771026 42515 start.go:128] duration metric: createHost completed in 11.685056264s
I0223 17:35:17.771044 42515 start.go:83] releasing machines lock for "old-k8s-version-977000", held for 11.685187325s
I0223 17:35:17.771140 42515 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-977000
I0223 17:35:17.827103 42515 ssh_runner.go:195] Run: cat /version.json
I0223 17:35:17.827119 42515 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0223 17:35:17.827177 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:17.827207 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:17.886823 42515 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61624 SSHKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa Username:docker}
I0223 17:35:17.886991 42515 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61624 SSHKeyPath:/Users/jenkins/minikube-integration/15909-24428/.minikube/machines/old-k8s-version-977000/id_rsa Username:docker}
I0223 17:35:18.227026 42515 ssh_runner.go:195] Run: systemctl --version
I0223 17:35:18.231940 42515 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0223 17:35:18.237000 42515 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0223 17:35:18.257029 42515 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0223 17:35:18.257107 42515 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0223 17:35:18.271235 42515 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0223 17:35:18.279142 42515 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0223 17:35:18.279155 42515 start.go:485] detecting cgroup driver to use...
I0223 17:35:18.279168 42515 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0223 17:35:18.279253 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0223 17:35:18.292733 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0223 17:35:18.301396 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0223 17:35:18.310185 42515 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0223 17:35:18.310245 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0223 17:35:18.318883 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0223 17:35:18.327551 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0223 17:35:18.335993 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0223 17:35:18.344514 42515 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0223 17:35:18.352647 42515 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0223 17:35:18.361362 42515 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0223 17:35:18.368822 42515 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0223 17:35:18.376103 42515 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0223 17:35:18.442475 42515 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0223 17:35:18.518693 42515 start.go:485] detecting cgroup driver to use...
I0223 17:35:18.518711 42515 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0223 17:35:18.518772 42515 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0223 17:35:18.529501 42515 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0223 17:35:18.529570 42515 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0223 17:35:18.539602 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0223 17:35:18.553963 42515 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0223 17:35:18.626054 42515 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0223 17:35:18.715120 42515 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0223 17:35:18.715137 42515 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0223 17:35:18.728566 42515 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0223 17:35:18.823182 42515 ssh_runner.go:195] Run: sudo systemctl restart docker
I0223 17:35:19.044247 42515 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0223 17:35:19.071062 42515 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0223 17:35:19.121063 42515 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
I0223 17:35:19.121218 42515 cli_runner.go:164] Run: docker exec -t old-k8s-version-977000 dig +short host.docker.internal
I0223 17:35:19.240245 42515 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0223 17:35:19.240359 42515 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0223 17:35:19.245011 42515 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0223 17:35:19.254961 42515 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-977000
I0223 17:35:19.312090 42515 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0223 17:35:19.312168 42515 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0223 17:35:19.331932 42515 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0223 17:35:19.331949 42515 docker.go:560] Images already preloaded, skipping extraction
I0223 17:35:19.332048 42515 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0223 17:35:19.352808 42515 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0223 17:35:19.352822 42515 cache_images.go:84] Images are preloaded, skipping loading
I0223 17:35:19.352916 42515 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0223 17:35:19.379389 42515 cni.go:84] Creating CNI manager for ""
I0223 17:35:19.379407 42515 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0223 17:35:19.379423 42515 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0223 17:35:19.379438 42515 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-977000 NodeName:old-k8s-version-977000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0223 17:35:19.379547 42515 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-977000"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-977000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0223 17:35:19.379621 42515 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-977000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-977000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0223 17:35:19.379681 42515 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0223 17:35:19.387571 42515 binaries.go:44] Found k8s binaries, skipping transfer
I0223 17:35:19.387631 42515 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0223 17:35:19.395162 42515 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0223 17:35:19.408213 42515 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0223 17:35:19.421335 42515 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0223 17:35:19.434723 42515 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0223 17:35:19.438828 42515 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0223 17:35:19.448744 42515 certs.go:56] Setting up /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000 for IP: 192.168.76.2
I0223 17:35:19.448760 42515 certs.go:186] acquiring lock for shared ca certs: {Name:mka4f8a2d0723293f88499f80fb83a53e78a6045 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.448945 42515 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.key
I0223 17:35:19.449007 42515 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15909-24428/.minikube/proxy-client-ca.key
I0223 17:35:19.449051 42515 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.key
I0223 17:35:19.449065 42515 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.crt with IP's: []
I0223 17:35:19.504927 42515 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.crt ...
I0223 17:35:19.504938 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.crt: {Name:mk25f5b99fd6f5d98a7ff9604e345f5d7a74ed89 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.510628 42515 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.key ...
I0223 17:35:19.510642 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/client.key: {Name:mk825e8e66f5ba3485b95521c8861d12fdee854f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.532707 42515 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key.31bdca25
I0223 17:35:19.532730 42515 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0223 17:35:19.682158 42515 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt.31bdca25 ...
I0223 17:35:19.682169 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt.31bdca25: {Name:mkb82aff22048118a7e5a3f763c71d474a315069 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.682437 42515 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key.31bdca25 ...
I0223 17:35:19.682455 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key.31bdca25: {Name:mk643651ff44dda1ccf3db3c6954109c0858c6fc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.682649 42515 certs.go:333] copying /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt
I0223 17:35:19.682818 42515 certs.go:337] copying /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key
I0223 17:35:19.682972 42515 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.key
I0223 17:35:19.682987 42515 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.crt with IP's: []
I0223 17:35:19.995443 42515 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.crt ...
I0223 17:35:19.995458 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.crt: {Name:mk93432dd4103fea3a1493f297bc718430a4ed85 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.995735 42515 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.key ...
I0223 17:35:19.995745 42515 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.key: {Name:mk38992f7a9000e57e23c188f76f18a7a0879237 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0223 17:35:19.996133 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/24885.pem (1338 bytes)
W0223 17:35:19.996183 42515 certs.go:397] ignoring /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/24885_empty.pem, impossibly tiny 0 bytes
I0223 17:35:19.996194 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca-key.pem (1675 bytes)
I0223 17:35:19.996230 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/ca.pem (1082 bytes)
I0223 17:35:19.996275 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/cert.pem (1123 bytes)
I0223 17:35:19.996307 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/certs/key.pem (1679 bytes)
I0223 17:35:19.996376 42515 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-24428/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15909-24428/.minikube/files/etc/ssl/certs/248852.pem (1708 bytes)
I0223 17:35:19.996863 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0223 17:35:20.015783 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0223 17:35:20.033928 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0223 17:35:20.051633 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/profiles/old-k8s-version-977000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0223 17:35:20.069251 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0223 17:35:20.086891 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0223 17:35:20.104570 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0223 17:35:20.122129 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0223 17:35:20.151706 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/files/etc/ssl/certs/248852.pem --> /usr/share/ca-certificates/248852.pem (1708 bytes)
I0223 17:35:20.171086 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0223 17:35:20.188585 42515 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-24428/.minikube/certs/24885.pem --> /usr/share/ca-certificates/24885.pem (1338 bytes)
I0223 17:35:20.205896 42515 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0223 17:35:20.219405 42515 ssh_runner.go:195] Run: openssl version
I0223 17:35:20.225063 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/248852.pem && ln -fs /usr/share/ca-certificates/248852.pem /etc/ssl/certs/248852.pem"
I0223 17:35:20.233328 42515 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/248852.pem
I0223 17:35:20.237359 42515 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Feb 24 00:46 /usr/share/ca-certificates/248852.pem
I0223 17:35:20.237407 42515 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/248852.pem
I0223 17:35:20.243052 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/248852.pem /etc/ssl/certs/3ec20f2e.0"
I0223 17:35:20.251393 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0223 17:35:20.259667 42515 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0223 17:35:20.263763 42515 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Feb 24 00:41 /usr/share/ca-certificates/minikubeCA.pem
I0223 17:35:20.263815 42515 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0223 17:35:20.269319 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0223 17:35:20.277484 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/24885.pem && ln -fs /usr/share/ca-certificates/24885.pem /etc/ssl/certs/24885.pem"
I0223 17:35:20.285572 42515 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/24885.pem
I0223 17:35:20.289596 42515 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Feb 24 00:46 /usr/share/ca-certificates/24885.pem
I0223 17:35:20.289648 42515 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/24885.pem
I0223 17:35:20.294965 42515 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/24885.pem /etc/ssl/certs/51391683.0"
I0223 17:35:20.303081 42515 kubeadm.go:401] StartCluster: {Name:old-k8s-version-977000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-977000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0223 17:35:20.303194 42515 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0223 17:35:20.322153 42515 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0223 17:35:20.330046 42515 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0223 17:35:20.337743 42515 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0223 17:35:20.337798 42515 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0223 17:35:20.345082 42515 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0223 17:35:20.345108 42515 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0223 17:35:20.393850 42515 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0223 17:35:20.393898 42515 kubeadm.go:322] [preflight] Running pre-flight checks
I0223 17:35:20.562940 42515 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0223 17:35:20.563013 42515 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0223 17:35:20.563090 42515 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0223 17:35:20.714621 42515 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0223 17:35:20.715399 42515 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0223 17:35:20.721681 42515 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0223 17:35:20.783446 42515 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0223 17:35:20.807017 42515 out.go:204] - Generating certificates and keys ...
I0223 17:35:20.807133 42515 kubeadm.go:322] [certs] Using existing ca certificate authority
I0223 17:35:20.807204 42515 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0223 17:35:20.914101 42515 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0223 17:35:21.035171 42515 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0223 17:35:21.159469 42515 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0223 17:35:21.438776 42515 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0223 17:35:21.707307 42515 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0223 17:35:21.707465 42515 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0223 17:35:21.909944 42515 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0223 17:35:21.910068 42515 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0223 17:35:21.949799 42515 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0223 17:35:22.059309 42515 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0223 17:35:22.210733 42515 kubeadm.go:322] [certs] Generating "sa" key and public key
I0223 17:35:22.210902 42515 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0223 17:35:22.299785 42515 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0223 17:35:22.450892 42515 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0223 17:35:22.666850 42515 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0223 17:35:22.802309 42515 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0223 17:35:22.802849 42515 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0223 17:35:22.824750 42515 out.go:204] - Booting up control plane ...
I0223 17:35:22.824865 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0223 17:35:22.824943 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0223 17:35:22.825001 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0223 17:35:22.825072 42515 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0223 17:35:22.825216 42515 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0223 17:36:02.812154 42515 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0223 17:36:02.813555 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:36:02.813787 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:36:07.815346 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:36:07.815582 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:36:17.815744 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:36:17.816284 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:36:37.818010 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:36:37.818270 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:37:17.848084 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:37:17.848335 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:37:17.848349 42515 kubeadm.go:322]
I0223 17:37:17.848387 42515 kubeadm.go:322] Unfortunately, an error has occurred:
I0223 17:37:17.848448 42515 kubeadm.go:322] timed out waiting for the condition
I0223 17:37:17.848467 42515 kubeadm.go:322]
I0223 17:37:17.848523 42515 kubeadm.go:322] This error is likely caused by:
I0223 17:37:17.848573 42515 kubeadm.go:322] - The kubelet is not running
I0223 17:37:17.848715 42515 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0223 17:37:17.848729 42515 kubeadm.go:322]
I0223 17:37:17.848861 42515 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0223 17:37:17.848901 42515 kubeadm.go:322] - 'systemctl status kubelet'
I0223 17:37:17.848939 42515 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0223 17:37:17.848947 42515 kubeadm.go:322]
I0223 17:37:17.849071 42515 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0223 17:37:17.849195 42515 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0223 17:37:17.849293 42515 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0223 17:37:17.849364 42515 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0223 17:37:17.849467 42515 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0223 17:37:17.849506 42515 kubeadm.go:322] - 'docker logs CONTAINERID'
I0223 17:37:17.851574 42515 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0223 17:37:17.851648 42515 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0223 17:37:17.851767 42515 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0223 17:37:17.851904 42515 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0223 17:37:17.851976 42515 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0223 17:37:17.852043 42515 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0223 17:37:17.852192 42515 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-977000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0223 17:37:17.852223 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0223 17:37:18.264398 42515 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0223 17:37:18.274426 42515 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0223 17:37:18.274490 42515 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0223 17:37:18.282141 42515 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0223 17:37:18.282169 42515 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0223 17:37:18.329445 42515 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0223 17:37:18.329489 42515 kubeadm.go:322] [preflight] Running pre-flight checks
I0223 17:37:18.494754 42515 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0223 17:37:18.494833 42515 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0223 17:37:18.494904 42515 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0223 17:37:18.648652 42515 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0223 17:37:18.649533 42515 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0223 17:37:18.656191 42515 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0223 17:37:18.733848 42515 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0223 17:37:18.755628 42515 out.go:204] - Generating certificates and keys ...
I0223 17:37:18.755759 42515 kubeadm.go:322] [certs] Using existing ca certificate authority
I0223 17:37:18.755837 42515 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0223 17:37:18.755917 42515 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0223 17:37:18.755973 42515 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0223 17:37:18.756052 42515 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0223 17:37:18.756106 42515 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0223 17:37:18.756180 42515 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0223 17:37:18.756250 42515 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0223 17:37:18.756323 42515 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0223 17:37:18.756406 42515 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0223 17:37:18.756464 42515 kubeadm.go:322] [certs] Using the existing "sa" key
I0223 17:37:18.756558 42515 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0223 17:37:18.796742 42515 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0223 17:37:18.942455 42515 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0223 17:37:19.161236 42515 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0223 17:37:19.257584 42515 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0223 17:37:19.258046 42515 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0223 17:37:19.282421 42515 out.go:204] - Booting up control plane ...
I0223 17:37:19.282575 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0223 17:37:19.282752 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0223 17:37:19.282927 42515 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0223 17:37:19.283079 42515 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0223 17:37:19.283370 42515 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0223 17:37:59.282295 42515 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0223 17:37:59.283180 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:37:59.283400 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:38:04.285287 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:38:04.285493 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:38:14.287696 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:38:14.287947 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:38:34.289998 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:38:34.290236 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:39:14.292823 42515 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0223 17:39:14.293040 42515 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0223 17:39:14.293051 42515 kubeadm.go:322]
I0223 17:39:14.293091 42515 kubeadm.go:322] Unfortunately, an error has occurred:
I0223 17:39:14.293148 42515 kubeadm.go:322] timed out waiting for the condition
I0223 17:39:14.293163 42515 kubeadm.go:322]
I0223 17:39:14.293201 42515 kubeadm.go:322] This error is likely caused by:
I0223 17:39:14.293249 42515 kubeadm.go:322] - The kubelet is not running
I0223 17:39:14.293364 42515 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0223 17:39:14.293377 42515 kubeadm.go:322]
I0223 17:39:14.293494 42515 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0223 17:39:14.293534 42515 kubeadm.go:322] - 'systemctl status kubelet'
I0223 17:39:14.293568 42515 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0223 17:39:14.293573 42515 kubeadm.go:322]
I0223 17:39:14.293705 42515 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0223 17:39:14.293822 42515 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0223 17:39:14.293923 42515 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0223 17:39:14.293989 42515 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0223 17:39:14.294078 42515 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0223 17:39:14.294139 42515 kubeadm.go:322] - 'docker logs CONTAINERID'
I0223 17:39:14.297062 42515 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0223 17:39:14.297143 42515 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0223 17:39:14.297248 42515 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0223 17:39:14.297326 42515 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0223 17:39:14.297391 42515 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0223 17:39:14.297451 42515 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0223 17:39:14.297478 42515 kubeadm.go:403] StartCluster complete in 3m53.946810607s
I0223 17:39:14.297570 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0223 17:39:14.318481 42515 logs.go:277] 0 containers: []
W0223 17:39:14.318496 42515 logs.go:279] No container was found matching "kube-apiserver"
I0223 17:39:14.318566 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0223 17:39:14.338951 42515 logs.go:277] 0 containers: []
W0223 17:39:14.338965 42515 logs.go:279] No container was found matching "etcd"
I0223 17:39:14.339038 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0223 17:39:14.359356 42515 logs.go:277] 0 containers: []
W0223 17:39:14.359369 42515 logs.go:279] No container was found matching "coredns"
I0223 17:39:14.359444 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0223 17:39:14.378941 42515 logs.go:277] 0 containers: []
W0223 17:39:14.378953 42515 logs.go:279] No container was found matching "kube-scheduler"
I0223 17:39:14.379023 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0223 17:39:14.398601 42515 logs.go:277] 0 containers: []
W0223 17:39:14.398612 42515 logs.go:279] No container was found matching "kube-proxy"
I0223 17:39:14.398682 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0223 17:39:14.418570 42515 logs.go:277] 0 containers: []
W0223 17:39:14.418584 42515 logs.go:279] No container was found matching "kube-controller-manager"
I0223 17:39:14.418663 42515 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0223 17:39:14.438334 42515 logs.go:277] 0 containers: []
W0223 17:39:14.438348 42515 logs.go:279] No container was found matching "kindnet"
I0223 17:39:14.438355 42515 logs.go:123] Gathering logs for dmesg ...
I0223 17:39:14.438364 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0223 17:39:14.450658 42515 logs.go:123] Gathering logs for describe nodes ...
I0223 17:39:14.450677 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0223 17:39:14.506165 42515 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0223 17:39:14.506176 42515 logs.go:123] Gathering logs for Docker ...
I0223 17:39:14.506183 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0223 17:39:14.530890 42515 logs.go:123] Gathering logs for container status ...
I0223 17:39:14.530904 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0223 17:39:16.576969 42515 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.046005625s)
I0223 17:39:16.577075 42515 logs.go:123] Gathering logs for kubelet ...
I0223 17:39:16.577083 42515 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0223 17:39:16.614326 42515 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 17:39:16.614346 42515 out.go:239] *
*
W0223 17:39:16.614449 42515 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 17:39:16.614462 42515 out.go:239] *
*
W0223 17:39:16.615053 42515 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0223 17:39:16.657672 42515 out.go:177]
W0223 17:39:16.716027 42515 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0223 17:39:16.716148 42515 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0223 17:39:16.716228 42515 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0223 17:39:16.778588 42515 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-977000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-977000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-977000:
-- stdout --
[
{
"Id": "821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c",
"Created": "2023-02-24T01:35:14.380880766Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 660961,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-02-24T01:35:14.709453937Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:b74f629d1852fc20b6085123d98944654faddf1d7e642b41aa2866d7a48081ea",
"ResolvConfPath": "/var/lib/docker/containers/821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c/hostname",
"HostsPath": "/var/lib/docker/containers/821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c/hosts",
"LogPath": "/var/lib/docker/containers/821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c/821104c1f595e8ccff9d72d82f90c0635818dc405623f087f556efcb01cc496c-json.log",
"Name": "/old-k8s-version-977000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-977000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-977000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/343c1e867eb65b62e125ebf4bcf5c863b2836597e553ec5b02be7ee9848cd16a-init/diff:/var/lib/docker/overlay2/e9788f80167b86b0aa5b795f2f3d600802b93fba8a2122959b7f1a31193d259a/diff:/var/lib/docker/overlay2/3311b08ed6fb82a6db41888762e507d536e3b26d2b03541b6a4147bee9325b04/diff:/var/lib/docker/overlay2/d791ec7693f308a6f9d9c24f7121f72e259d6a91f76cb17bb6aa879ac0aaf9e7/diff:/var/lib/docker/overlay2/68f7de26db177ab2eeec7f9dbb42f6eda85ec36655cbdfed5ee15aba180eb346/diff:/var/lib/docker/overlay2/f44dd4a4bdd627648a1d240924fcb811136db11f9c23529924b5eb0ad38b341d/diff:/var/lib/docker/overlay2/62fad3781f53b4c81d63dfbeb52c72fd61caa73bafe8d8765482ced0b72e52b5/diff:/var/lib/docker/overlay2/0365eeefae207274beb52bb15213d5ad6afabdfa6d616f6a50fd835379a6e1d9/diff:/var/lib/docker/overlay2/af460618566d5be40a633830ec0cca15769c940e94c046541f4a327299f97665/diff:/var/lib/docker/overlay2/4c27ffc65a78b664cce78bc88f4bd2dd69a17ec0e06fcf2700affcab1297be85/diff:/var/lib/docker/overlay2/e5792b
fb35e95c8e01826dcc8dfcd7a6f67883c9cef1bfcce8ae6a6e575de809/diff:/var/lib/docker/overlay2/495c698be46c617acf68fa28ebe4664feb6edb806c90a3c88c8fc3311d2fff64/diff:/var/lib/docker/overlay2/0db5db3040241866674245317d2a38eb48acdd50c6b48b8feb278ee40ab20e13/diff:/var/lib/docker/overlay2/c302751df21df9b1af14da38e239c038d79560c4a9daa0e4e3af41516c22bb8e/diff:/var/lib/docker/overlay2/651c97220cad2e4a3e41428005d1d9376b60523dc754410b324390527fab638b/diff:/var/lib/docker/overlay2/ffeb543640423404b334d309a5c4e31596a2752398132b7f4d271f82d44b7c99/diff:/var/lib/docker/overlay2/3bad591675b09dceceb973e57645a11765029010b632167351246799a7133e16/diff:/var/lib/docker/overlay2/5d3e5b2318b8a82a24c46f96b95b036cda2ece4fa0e95188ae19124d588a5ca3/diff:/var/lib/docker/overlay2/a265e8ad9fc624a094ebc7ca4bdf43fdb2c52c8ba3898d4f2c32e84befe7318d/diff:/var/lib/docker/overlay2/346c14b8ef206041c9e466c112208098b1ebe602d9ac6094edc575a3425fa283/diff:/var/lib/docker/overlay2/f5a551deafb80cde81876b6c1f2e741884a1cf1cdad3fb5d6ffab113f31e4a0a/diff:/var/lib/d
ocker/overlay2/8db1dfd60158d30288740d04e98225a6a77b9f497fbc6370795f32d48210529c/diff:/var/lib/docker/overlay2/b9f3f954350246b0dad2d56d0aaec84a79c9be55d927d07f4161981ae43d0ace/diff:/var/lib/docker/overlay2/efd37a9d5708c937817b9b909ff0f80e5992b2b4a5237e6395dd3aeb837ed3ff/diff:/var/lib/docker/overlay2/ee9d652a4f5cf4055607be0c895984aa253e4b5c6dd00414087ccdedd5329759/diff:/var/lib/docker/overlay2/9bebcd890fe3dd6c34e8793f81aa69fe76a693e4c7420d26948bc60054d3861e/diff:/var/lib/docker/overlay2/bcbf2175a046d7d3541babcb252fa51c3abb433fb894ff21c8cda4dc51df3342/diff:/var/lib/docker/overlay2/7e8d9a7a9617277b163113b911112644d8d12676a1dab919f15589cd82f326f6/diff:/var/lib/docker/overlay2/940f18fbfcb7ca9943941561a17fdf3ff24754a5a3b232c5096cee47c3dc686d/diff:/var/lib/docker/overlay2/706967ddab541e8e1c0ba8458a02526f8c6fdd7e22d9fdc2709f3f8091926e70/diff:/var/lib/docker/overlay2/89624b64a8c5f018614f425104936164ee53bd0d89163c1e2bb5ca591b7ea41f/diff:/var/lib/docker/overlay2/04cbeaf84433f334943d8d99476d687f35576e2f1bb2b9485f6406b0501
58eaf/diff:/var/lib/docker/overlay2/bb646746ad25b161f2ffecc9a438a557fff814a059e3d4fbb51f99af03e2b084/diff:/var/lib/docker/overlay2/2592092232a2ae41548bbdbc384ae0d335e680e68ba38cbe462b3dc648c2c3b4/diff:/var/lib/docker/overlay2/e3d395e2c69e44157aa14eedcc88e174800ec7eb6658ea0591ab50961cd89577/diff:/var/lib/docker/overlay2/660018fe497ca74325f295cce62d29ce6fc01008cd1529da22e979e3e8d87582/diff:/var/lib/docker/overlay2/8490879fff1ac447e19bda3843b477926744c3d2a8bc0a22264ae9b30c1ba386/diff:/var/lib/docker/overlay2/3cb21451fde458b4258fdb2d67f2a8557d9f550b0c6de008eda8092896cb10fa/diff:/var/lib/docker/overlay2/6d00d6a43bfd7d84f5c225c980f2c0a77ab7a60911980c6fb6369e860e1e2649/diff:/var/lib/docker/overlay2/4592e35c9c34141b86e00739ab5c223ec86f25dab6561a52179dc171be01eb12/diff:/var/lib/docker/overlay2/52c10afcc404325c79396d496762927c9351c2f6b462fb6e02892d4b489d2725/diff:/var/lib/docker/overlay2/20ec2a2a295d10c67a08e8a7263279eb08ed879417c5a36d41c78ea9dcc0cefb/diff:/var/lib/docker/overlay2/ba42e575768c7e5e87f54e82ba3e2a318e3271
bdfa23bce99c7637ebefc0d0ba/diff",
"MergedDir": "/var/lib/docker/overlay2/343c1e867eb65b62e125ebf4bcf5c863b2836597e553ec5b02be7ee9848cd16a/merged",
"UpperDir": "/var/lib/docker/overlay2/343c1e867eb65b62e125ebf4bcf5c863b2836597e553ec5b02be7ee9848cd16a/diff",
"WorkDir": "/var/lib/docker/overlay2/343c1e867eb65b62e125ebf4bcf5c863b2836597e553ec5b02be7ee9848cd16a/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-977000",
"Source": "/var/lib/docker/volumes/old-k8s-version-977000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-977000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-977000",
"name.minikube.sigs.k8s.io": "old-k8s-version-977000",
"org.opencontainers.image.ref.name": "ubuntu",
"org.opencontainers.image.version": "20.04",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "3505e397ab80e9420d900918aec6e67a14534455d9486eac92fb062a100dc5c7",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61624"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61625"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61626"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61627"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61628"
}
]
},
"SandboxKey": "/var/run/docker/netns/3505e397ab80",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-977000": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"821104c1f595",
"old-k8s-version-977000"
],
"NetworkID": "b62760f23c198a189d083cf13177930b5af19fc8f1e171a0fb08c0832e6d6e8a",
"EndpointID": "5d0c9d0869863374e75fabba63aa5547da0714b9a5633a5f32972176a42d8c14",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-977000 -n old-k8s-version-977000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-977000 -n old-k8s-version-977000: exit status 6 (398.155338ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0223 17:39:17.304935 43367 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-977000" does not appear in /Users/jenkins/minikube-integration/15909-24428/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-977000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (252.20s)