=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-583000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-583000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m10.315228553s)
-- stdout --
* [old-k8s-version-583000] minikube v1.29.0 on Darwin 13.2.1
- MINIKUBE_LOCATION=15909
- KUBECONFIG=/Users/jenkins/minikube-integration/15909-26406/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-26406/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-583000 in cluster old-k8s-version-583000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0224 15:39:18.772256 44716 out.go:296] Setting OutFile to fd 1 ...
I0224 15:39:18.772423 44716 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0224 15:39:18.772428 44716 out.go:309] Setting ErrFile to fd 2...
I0224 15:39:18.772433 44716 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0224 15:39:18.772535 44716 root.go:336] Updating PATH: /Users/jenkins/minikube-integration/15909-26406/.minikube/bin
I0224 15:39:18.773956 44716 out.go:303] Setting JSON to false
I0224 15:39:18.792375 44716 start.go:125] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":9532,"bootTime":1677272426,"procs":383,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.2.1","kernelVersion":"22.3.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W0224 15:39:18.792471 44716 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0224 15:39:18.813592 44716 out.go:177] * [old-k8s-version-583000] minikube v1.29.0 on Darwin 13.2.1
I0224 15:39:18.855633 44716 out.go:177] - MINIKUBE_LOCATION=15909
I0224 15:39:18.855674 44716 notify.go:220] Checking for updates...
I0224 15:39:18.899754 44716 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15909-26406/kubeconfig
I0224 15:39:18.921661 44716 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0224 15:39:18.942514 44716 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0224 15:39:18.963864 44716 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15909-26406/.minikube
I0224 15:39:18.985520 44716 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0224 15:39:19.007465 44716 config.go:182] Loaded profile config "false-416000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0224 15:39:19.007568 44716 driver.go:365] Setting default libvirt URI to qemu:///system
I0224 15:39:19.070019 44716 docker.go:121] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0224 15:39:19.070180 44716 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0224 15:39:19.212113 44716 info.go:266] docker info: {ID:IP4W:MU7T:LXXP:A5FK:AZDS:VO26:5WXJ:AUD6:DYFY:LVPL:GBAL:LED3 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:59 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-24 23:39:19.119625107 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0224 15:39:19.255636 44716 out.go:177] * Using the docker driver based on user configuration
I0224 15:39:19.276575 44716 start.go:296] selected driver: docker
I0224 15:39:19.276601 44716 start.go:857] validating driver "docker" against <nil>
I0224 15:39:19.276618 44716 start.go:868] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0224 15:39:19.280932 44716 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0224 15:39:19.424184 44716 info.go:266] docker info: {ID:IP4W:MU7T:LXXP:A5FK:AZDS:VO26:5WXJ:AUD6:DYFY:LVPL:GBAL:LED3 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:59 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-24 23:39:19.331470603 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0224 15:39:19.424303 44716 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0224 15:39:19.424481 44716 start_flags.go:919] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0224 15:39:19.446287 44716 out.go:177] * Using Docker Desktop driver with root privileges
I0224 15:39:19.467806 44716 cni.go:84] Creating CNI manager for ""
I0224 15:39:19.467914 44716 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0224 15:39:19.467946 44716 start_flags.go:319] config:
{Name:old-k8s-version-583000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-583000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0224 15:39:19.511966 44716 out.go:177] * Starting control plane node old-k8s-version-583000 in cluster old-k8s-version-583000
I0224 15:39:19.532880 44716 cache.go:120] Beginning downloading kic base image for docker with docker
I0224 15:39:19.553952 44716 out.go:177] * Pulling base image ...
I0224 15:39:19.595920 44716 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0224 15:39:19.595976 44716 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon
I0224 15:39:19.596016 44716 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15909-26406/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0224 15:39:19.596036 44716 cache.go:57] Caching tarball of preloaded images
I0224 15:39:19.596227 44716 preload.go:174] Found /Users/jenkins/minikube-integration/15909-26406/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0224 15:39:19.596246 44716 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0224 15:39:19.597247 44716 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/config.json ...
I0224 15:39:19.597417 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/config.json: {Name:mkd2607386d03742988084ec03dff342ad825c13 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:19.653580 44716 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc in local docker daemon, skipping pull
I0224 15:39:19.653823 44716 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc exists in daemon, skipping load
I0224 15:39:19.653843 44716 cache.go:193] Successfully downloaded all kic artifacts
I0224 15:39:19.653892 44716 start.go:364] acquiring machines lock for old-k8s-version-583000: {Name:mk9aaddc56a14fcb74e6153f904a41eee9f24006 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0224 15:39:19.654038 44716 start.go:368] acquired machines lock for "old-k8s-version-583000" in 134.828µs
I0224 15:39:19.654066 44716 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-583000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-583000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0224 15:39:19.654149 44716 start.go:125] createHost starting for "" (driver="docker")
I0224 15:39:19.675894 44716 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0224 15:39:19.676263 44716 start.go:159] libmachine.API.Create for "old-k8s-version-583000" (driver="docker")
I0224 15:39:19.676304 44716 client.go:168] LocalClient.Create starting
I0224 15:39:19.676551 44716 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem
I0224 15:39:19.676647 44716 main.go:141] libmachine: Decoding PEM data...
I0224 15:39:19.676683 44716 main.go:141] libmachine: Parsing certificate...
I0224 15:39:19.676801 44716 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/cert.pem
I0224 15:39:19.676870 44716 main.go:141] libmachine: Decoding PEM data...
I0224 15:39:19.676886 44716 main.go:141] libmachine: Parsing certificate...
I0224 15:39:19.677707 44716 cli_runner.go:164] Run: docker network inspect old-k8s-version-583000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0224 15:39:19.733197 44716 cli_runner.go:211] docker network inspect old-k8s-version-583000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0224 15:39:19.733297 44716 network_create.go:281] running [docker network inspect old-k8s-version-583000] to gather additional debugging logs...
I0224 15:39:19.733309 44716 cli_runner.go:164] Run: docker network inspect old-k8s-version-583000
W0224 15:39:19.788385 44716 cli_runner.go:211] docker network inspect old-k8s-version-583000 returned with exit code 1
I0224 15:39:19.788414 44716 network_create.go:284] error running [docker network inspect old-k8s-version-583000]: docker network inspect old-k8s-version-583000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-583000
I0224 15:39:19.788427 44716 network_create.go:286] output of [docker network inspect old-k8s-version-583000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-583000
** /stderr **
I0224 15:39:19.788507 44716 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0224 15:39:19.844793 44716 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0224 15:39:19.846208 44716 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0224 15:39:19.847741 44716 network.go:212] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0224 15:39:19.848050 44716 network.go:209] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0005a81f0}
I0224 15:39:19.848063 44716 network_create.go:123] attempt to create docker network old-k8s-version-583000 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0224 15:39:19.848129 44716 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-583000 old-k8s-version-583000
I0224 15:39:19.936288 44716 network_create.go:107] docker network old-k8s-version-583000 192.168.76.0/24 created
I0224 15:39:19.936320 44716 kic.go:117] calculated static IP "192.168.76.2" for the "old-k8s-version-583000" container
I0224 15:39:19.936476 44716 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0224 15:39:19.993330 44716 cli_runner.go:164] Run: docker volume create old-k8s-version-583000 --label name.minikube.sigs.k8s.io=old-k8s-version-583000 --label created_by.minikube.sigs.k8s.io=true
I0224 15:39:20.049024 44716 oci.go:103] Successfully created a docker volume old-k8s-version-583000
I0224 15:39:20.049166 44716 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-583000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-583000 --entrypoint /usr/bin/test -v old-k8s-version-583000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -d /var/lib
I0224 15:39:20.595655 44716 oci.go:107] Successfully prepared a docker volume old-k8s-version-583000
I0224 15:39:20.595685 44716 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0224 15:39:20.595699 44716 kic.go:190] Starting extracting preloaded images to volume ...
I0224 15:39:20.595827 44716 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-26406/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-583000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir
I0224 15:39:27.133253 44716 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15909-26406/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-583000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc -I lz4 -xf /preloaded.tar -C /extractDir: (6.537323741s)
I0224 15:39:27.133275 44716 kic.go:199] duration metric: took 6.537518 seconds to extract preloaded images to volume
I0224 15:39:27.133393 44716 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0224 15:39:27.279207 44716 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-583000 --name old-k8s-version-583000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-583000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-583000 --network old-k8s-version-583000 --ip 192.168.76.2 --volume old-k8s-version-583000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc
I0224 15:39:27.658004 44716 cli_runner.go:164] Run: docker container inspect old-k8s-version-583000 --format={{.State.Running}}
I0224 15:39:27.750096 44716 cli_runner.go:164] Run: docker container inspect old-k8s-version-583000 --format={{.State.Status}}
I0224 15:39:27.812860 44716 cli_runner.go:164] Run: docker exec old-k8s-version-583000 stat /var/lib/dpkg/alternatives/iptables
I0224 15:39:27.926089 44716 oci.go:144] the created container "old-k8s-version-583000" has a running status.
I0224 15:39:27.926117 44716 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa...
I0224 15:39:27.970887 44716 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0224 15:39:28.086432 44716 cli_runner.go:164] Run: docker container inspect old-k8s-version-583000 --format={{.State.Status}}
I0224 15:39:28.146617 44716 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0224 15:39:28.146644 44716 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-583000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0224 15:39:28.257236 44716 cli_runner.go:164] Run: docker container inspect old-k8s-version-583000 --format={{.State.Status}}
I0224 15:39:28.315429 44716 machine.go:88] provisioning docker machine ...
I0224 15:39:28.315475 44716 ubuntu.go:169] provisioning hostname "old-k8s-version-583000"
I0224 15:39:28.315577 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:28.373234 44716 main.go:141] libmachine: Using SSH client type: native
I0224 15:39:28.373626 44716 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61528 <nil> <nil>}
I0224 15:39:28.373643 44716 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-583000 && echo "old-k8s-version-583000" | sudo tee /etc/hostname
I0224 15:39:28.517718 44716 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-583000
I0224 15:39:28.517811 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:28.575988 44716 main.go:141] libmachine: Using SSH client type: native
I0224 15:39:28.576352 44716 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61528 <nil> <nil>}
I0224 15:39:28.576367 44716 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-583000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-583000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-583000' | sudo tee -a /etc/hosts;
fi
fi
I0224 15:39:28.711013 44716 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0224 15:39:28.711038 44716 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15909-26406/.minikube CaCertPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15909-26406/.minikube}
I0224 15:39:28.711057 44716 ubuntu.go:177] setting up certificates
I0224 15:39:28.711068 44716 provision.go:83] configureAuth start
I0224 15:39:28.711154 44716 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-583000
I0224 15:39:28.769026 44716 provision.go:138] copyHostCerts
I0224 15:39:28.769119 44716 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.pem, removing ...
I0224 15:39:28.769128 44716 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.pem
I0224 15:39:28.769252 44716 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.pem (1078 bytes)
I0224 15:39:28.769452 44716 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-26406/.minikube/cert.pem, removing ...
I0224 15:39:28.769465 44716 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-26406/.minikube/cert.pem
I0224 15:39:28.769533 44716 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15909-26406/.minikube/cert.pem (1123 bytes)
I0224 15:39:28.769688 44716 exec_runner.go:144] found /Users/jenkins/minikube-integration/15909-26406/.minikube/key.pem, removing ...
I0224 15:39:28.769702 44716 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15909-26406/.minikube/key.pem
I0224 15:39:28.769765 44716 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15909-26406/.minikube/key.pem (1675 bytes)
I0224 15:39:28.769889 44716 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-583000 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-583000]
I0224 15:39:28.909406 44716 provision.go:172] copyRemoteCerts
I0224 15:39:28.909468 44716 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0224 15:39:28.909521 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:28.967525 44716 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61528 SSHKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa Username:docker}
I0224 15:39:29.062713 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0224 15:39:29.080605 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0224 15:39:29.097767 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0224 15:39:29.115099 44716 provision.go:86] duration metric: configureAuth took 404.014024ms
I0224 15:39:29.115113 44716 ubuntu.go:193] setting minikube options for container-runtime
I0224 15:39:29.115270 44716 config.go:182] Loaded profile config "old-k8s-version-583000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0224 15:39:29.115338 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:29.174072 44716 main.go:141] libmachine: Using SSH client type: native
I0224 15:39:29.174435 44716 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61528 <nil> <nil>}
I0224 15:39:29.174448 44716 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0224 15:39:29.309668 44716 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0224 15:39:29.309688 44716 ubuntu.go:71] root file system type: overlay
I0224 15:39:29.309789 44716 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0224 15:39:29.309871 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:29.369414 44716 main.go:141] libmachine: Using SSH client type: native
I0224 15:39:29.369779 44716 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61528 <nil> <nil>}
I0224 15:39:29.369833 44716 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0224 15:39:29.512976 44716 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0224 15:39:29.513088 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:29.571485 44716 main.go:141] libmachine: Using SSH client type: native
I0224 15:39:29.571846 44716 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140f720] 0x1412660 <nil> [] 0s} 127.0.0.1 61528 <nil> <nil>}
I0224 15:39:29.571858 44716 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0224 15:39:30.234681 44716 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-02-09 19:46:56.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-02-24 23:39:29.510425235 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0224 15:39:30.234703 44716 machine.go:91] provisioned docker machine in 1.919235991s
I0224 15:39:30.234709 44716 client.go:171] LocalClient.Create took 10.558300262s
I0224 15:39:30.234727 44716 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-583000" took 10.558371888s
I0224 15:39:30.234735 44716 start.go:300] post-start starting for "old-k8s-version-583000" (driver="docker")
I0224 15:39:30.234739 44716 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0224 15:39:30.234815 44716 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0224 15:39:30.234870 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:30.294036 44716 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61528 SSHKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa Username:docker}
I0224 15:39:30.388952 44716 ssh_runner.go:195] Run: cat /etc/os-release
I0224 15:39:30.392543 44716 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0224 15:39:30.392562 44716 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0224 15:39:30.392569 44716 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0224 15:39:30.392574 44716 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0224 15:39:30.392585 44716 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-26406/.minikube/addons for local assets ...
I0224 15:39:30.392683 44716 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15909-26406/.minikube/files for local assets ...
I0224 15:39:30.392860 44716 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15909-26406/.minikube/files/etc/ssl/certs/268712.pem -> 268712.pem in /etc/ssl/certs
I0224 15:39:30.393073 44716 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0224 15:39:30.400314 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/files/etc/ssl/certs/268712.pem --> /etc/ssl/certs/268712.pem (1708 bytes)
I0224 15:39:30.417887 44716 start.go:303] post-start completed in 183.142612ms
I0224 15:39:30.418429 44716 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-583000
I0224 15:39:30.477073 44716 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/config.json ...
I0224 15:39:30.477499 44716 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0224 15:39:30.477566 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:30.534859 44716 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61528 SSHKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa Username:docker}
I0224 15:39:30.627041 44716 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0224 15:39:30.632142 44716 start.go:128] duration metric: createHost completed in 10.977881782s
I0224 15:39:30.632167 44716 start.go:83] releasing machines lock for "old-k8s-version-583000", held for 10.978021631s
I0224 15:39:30.632283 44716 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-583000
I0224 15:39:30.689940 44716 ssh_runner.go:195] Run: cat /version.json
I0224 15:39:30.689944 44716 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0224 15:39:30.690017 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:30.690033 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:30.750560 44716 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61528 SSHKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa Username:docker}
I0224 15:39:30.751135 44716 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61528 SSHKeyPath:/Users/jenkins/minikube-integration/15909-26406/.minikube/machines/old-k8s-version-583000/id_rsa Username:docker}
I0224 15:39:30.840458 44716 ssh_runner.go:195] Run: systemctl --version
I0224 15:39:31.142014 44716 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0224 15:39:31.147293 44716 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0224 15:39:31.167610 44716 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0224 15:39:31.167684 44716 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0224 15:39:31.181514 44716 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0224 15:39:31.189371 44716 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0224 15:39:31.189387 44716 start.go:485] detecting cgroup driver to use...
I0224 15:39:31.189397 44716 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0224 15:39:31.189476 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0224 15:39:31.203005 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0224 15:39:31.211553 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0224 15:39:31.220749 44716 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0224 15:39:31.220815 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0224 15:39:31.229481 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0224 15:39:31.238080 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0224 15:39:31.246795 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0224 15:39:31.255448 44716 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0224 15:39:31.263328 44716 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0224 15:39:31.271778 44716 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0224 15:39:31.279382 44716 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0224 15:39:31.287456 44716 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0224 15:39:31.352468 44716 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0224 15:39:31.420875 44716 start.go:485] detecting cgroup driver to use...
I0224 15:39:31.420895 44716 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0224 15:39:31.420961 44716 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0224 15:39:31.431589 44716 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0224 15:39:31.431647 44716 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0224 15:39:31.442239 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0224 15:39:31.456387 44716 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0224 15:39:31.552569 44716 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0224 15:39:31.614043 44716 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0224 15:39:31.614063 44716 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0224 15:39:31.649371 44716 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0224 15:39:31.710327 44716 ssh_runner.go:195] Run: sudo systemctl restart docker
I0224 15:39:31.955031 44716 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0224 15:39:31.981177 44716 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0224 15:39:32.048668 44716 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 23.0.1 ...
I0224 15:39:32.048854 44716 cli_runner.go:164] Run: docker exec -t old-k8s-version-583000 dig +short host.docker.internal
I0224 15:39:32.168801 44716 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0224 15:39:32.168917 44716 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0224 15:39:32.173340 44716 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0224 15:39:32.183573 44716 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-583000
I0224 15:39:32.243184 44716 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0224 15:39:32.243273 44716 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0224 15:39:32.264206 44716 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0224 15:39:32.264229 44716 docker.go:560] Images already preloaded, skipping extraction
I0224 15:39:32.264301 44716 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0224 15:39:32.285296 44716 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0224 15:39:32.285310 44716 cache_images.go:84] Images are preloaded, skipping loading
I0224 15:39:32.285415 44716 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0224 15:39:32.312655 44716 cni.go:84] Creating CNI manager for ""
I0224 15:39:32.312674 44716 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0224 15:39:32.312689 44716 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0224 15:39:32.312715 44716 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-583000 NodeName:old-k8s-version-583000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0224 15:39:32.312836 44716 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-583000"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-583000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0224 15:39:32.312914 44716 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-583000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-583000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0224 15:39:32.312981 44716 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0224 15:39:32.321063 44716 binaries.go:44] Found k8s binaries, skipping transfer
I0224 15:39:32.321124 44716 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0224 15:39:32.329008 44716 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0224 15:39:32.342113 44716 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0224 15:39:32.355017 44716 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0224 15:39:32.368209 44716 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0224 15:39:32.372091 44716 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0224 15:39:32.381972 44716 certs.go:56] Setting up /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000 for IP: 192.168.76.2
I0224 15:39:32.381989 44716 certs.go:186] acquiring lock for shared ca certs: {Name:mkabe8f97a4ffa8384a45c3fc6225c6b2025baa8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.382169 44716 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.key
I0224 15:39:32.382252 44716 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15909-26406/.minikube/proxy-client-ca.key
I0224 15:39:32.382300 44716 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.key
I0224 15:39:32.382317 44716 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.crt with IP's: []
I0224 15:39:32.465165 44716 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.crt ...
I0224 15:39:32.465177 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.crt: {Name:mk55f60cdd892f02ffc5c80be721a757a13b0afa Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.465493 44716 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.key ...
I0224 15:39:32.465501 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/client.key: {Name:mkc5b509d21f9d1d254d5a041a75a47ff5e10c12 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.465698 44716 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key.31bdca25
I0224 15:39:32.465713 44716 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0224 15:39:32.504086 44716 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt.31bdca25 ...
I0224 15:39:32.504094 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt.31bdca25: {Name:mkd92244873505dbdd962df06228125f047fd833 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.504311 44716 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key.31bdca25 ...
I0224 15:39:32.504318 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key.31bdca25: {Name:mk7eb9547414c976ecb1c411d54d68eff28e95ca Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.504508 44716 certs.go:333] copying /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt
I0224 15:39:32.504685 44716 certs.go:337] copying /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key
I0224 15:39:32.504856 44716 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.key
I0224 15:39:32.504871 44716 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.crt with IP's: []
I0224 15:39:32.642231 44716 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.crt ...
I0224 15:39:32.642245 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.crt: {Name:mkc6137b1a44f8754442584ad219305cb965fa48 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.642531 44716 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.key ...
I0224 15:39:32.642544 44716 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.key: {Name:mk4f693d8816763adf74ed676ae7fccabb433ae3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0224 15:39:32.643011 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/26871.pem (1338 bytes)
W0224 15:39:32.643065 44716 certs.go:397] ignoring /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/26871_empty.pem, impossibly tiny 0 bytes
I0224 15:39:32.643079 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca-key.pem (1679 bytes)
I0224 15:39:32.643114 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/ca.pem (1078 bytes)
I0224 15:39:32.643148 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/cert.pem (1123 bytes)
I0224 15:39:32.643181 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/certs/key.pem (1675 bytes)
I0224 15:39:32.643255 44716 certs.go:401] found cert: /Users/jenkins/minikube-integration/15909-26406/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15909-26406/.minikube/files/etc/ssl/certs/268712.pem (1708 bytes)
I0224 15:39:32.643789 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0224 15:39:32.662228 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0224 15:39:32.679761 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0224 15:39:32.697184 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/profiles/old-k8s-version-583000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0224 15:39:32.714582 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0224 15:39:32.731914 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0224 15:39:32.749237 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0224 15:39:32.766706 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0224 15:39:32.784449 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/files/etc/ssl/certs/268712.pem --> /usr/share/ca-certificates/268712.pem (1708 bytes)
I0224 15:39:32.801861 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0224 15:39:32.820112 44716 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15909-26406/.minikube/certs/26871.pem --> /usr/share/ca-certificates/26871.pem (1338 bytes)
I0224 15:39:32.837701 44716 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0224 15:39:32.850678 44716 ssh_runner.go:195] Run: openssl version
I0224 15:39:32.856493 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/268712.pem && ln -fs /usr/share/ca-certificates/268712.pem /etc/ssl/certs/268712.pem"
I0224 15:39:32.865097 44716 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/268712.pem
I0224 15:39:32.869086 44716 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Feb 24 22:48 /usr/share/ca-certificates/268712.pem
I0224 15:39:32.869130 44716 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/268712.pem
I0224 15:39:32.874857 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/268712.pem /etc/ssl/certs/3ec20f2e.0"
I0224 15:39:32.883287 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0224 15:39:32.891885 44716 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0224 15:39:32.896312 44716 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Feb 24 22:42 /usr/share/ca-certificates/minikubeCA.pem
I0224 15:39:32.896364 44716 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0224 15:39:32.902200 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0224 15:39:32.910740 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/26871.pem && ln -fs /usr/share/ca-certificates/26871.pem /etc/ssl/certs/26871.pem"
I0224 15:39:32.918994 44716 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/26871.pem
I0224 15:39:32.923047 44716 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Feb 24 22:48 /usr/share/ca-certificates/26871.pem
I0224 15:39:32.923098 44716 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/26871.pem
I0224 15:39:32.928633 44716 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/26871.pem /etc/ssl/certs/51391683.0"
I0224 15:39:32.936994 44716 kubeadm.go:401] StartCluster: {Name:old-k8s-version-583000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-583000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0224 15:39:32.937101 44716 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0224 15:39:32.956607 44716 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0224 15:39:32.964669 44716 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0224 15:39:32.972225 44716 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0224 15:39:32.972288 44716 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0224 15:39:32.980055 44716 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0224 15:39:32.980094 44716 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0224 15:39:33.029835 44716 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0224 15:39:33.030014 44716 kubeadm.go:322] [preflight] Running pre-flight checks
I0224 15:39:33.200427 44716 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0224 15:39:33.200512 44716 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0224 15:39:33.200597 44716 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0224 15:39:33.354986 44716 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0224 15:39:33.355737 44716 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0224 15:39:33.362227 44716 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0224 15:39:33.426682 44716 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0224 15:39:33.448339 44716 out.go:204] - Generating certificates and keys ...
I0224 15:39:33.448425 44716 kubeadm.go:322] [certs] Using existing ca certificate authority
I0224 15:39:33.448496 44716 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0224 15:39:33.616334 44716 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0224 15:39:33.740341 44716 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0224 15:39:33.828924 44716 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0224 15:39:33.957212 44716 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0224 15:39:34.006794 44716 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0224 15:39:34.006916 44716 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0224 15:39:34.055490 44716 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0224 15:39:34.055736 44716 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0224 15:39:34.197316 44716 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0224 15:39:34.285624 44716 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0224 15:39:34.453801 44716 kubeadm.go:322] [certs] Generating "sa" key and public key
I0224 15:39:34.454122 44716 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0224 15:39:34.542554 44716 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0224 15:39:34.661938 44716 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0224 15:39:34.737368 44716 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0224 15:39:34.811451 44716 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0224 15:39:34.812329 44716 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0224 15:39:34.835745 44716 out.go:204] - Booting up control plane ...
I0224 15:39:34.835918 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0224 15:39:34.836009 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0224 15:39:34.836099 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0224 15:39:34.836176 44716 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0224 15:39:34.836334 44716 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0224 15:40:14.821218 44716 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0224 15:40:14.821407 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:40:14.821766 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:40:19.823167 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:40:19.823381 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:40:29.824079 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:40:29.824239 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:40:49.826193 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:40:49.826420 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:41:29.828374 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:41:29.828588 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:41:29.828602 44716 kubeadm.go:322]
I0224 15:41:29.828639 44716 kubeadm.go:322] Unfortunately, an error has occurred:
I0224 15:41:29.828687 44716 kubeadm.go:322] timed out waiting for the condition
I0224 15:41:29.828702 44716 kubeadm.go:322]
I0224 15:41:29.828734 44716 kubeadm.go:322] This error is likely caused by:
I0224 15:41:29.828767 44716 kubeadm.go:322] - The kubelet is not running
I0224 15:41:29.828875 44716 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0224 15:41:29.828887 44716 kubeadm.go:322]
I0224 15:41:29.828995 44716 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0224 15:41:29.829040 44716 kubeadm.go:322] - 'systemctl status kubelet'
I0224 15:41:29.829076 44716 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0224 15:41:29.829082 44716 kubeadm.go:322]
I0224 15:41:29.829255 44716 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0224 15:41:29.829361 44716 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0224 15:41:29.829442 44716 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0224 15:41:29.829500 44716 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0224 15:41:29.829595 44716 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0224 15:41:29.829631 44716 kubeadm.go:322] - 'docker logs CONTAINERID'
I0224 15:41:29.832036 44716 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0224 15:41:29.832099 44716 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0224 15:41:29.832202 44716 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0224 15:41:29.832315 44716 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0224 15:41:29.832384 44716 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0224 15:41:29.832451 44716 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0224 15:41:29.832647 44716 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-583000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0224 15:41:29.832688 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0224 15:41:30.243600 44716 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0224 15:41:30.253549 44716 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0224 15:41:30.253604 44716 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0224 15:41:30.261256 44716 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0224 15:41:30.261281 44716 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0224 15:41:30.311331 44716 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0224 15:41:30.311375 44716 kubeadm.go:322] [preflight] Running pre-flight checks
I0224 15:41:30.480631 44716 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0224 15:41:30.480740 44716 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0224 15:41:30.480840 44716 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0224 15:41:30.634337 44716 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0224 15:41:30.634902 44716 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0224 15:41:30.641531 44716 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0224 15:41:30.707719 44716 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0224 15:41:30.729352 44716 out.go:204] - Generating certificates and keys ...
I0224 15:41:30.729433 44716 kubeadm.go:322] [certs] Using existing ca certificate authority
I0224 15:41:30.729501 44716 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0224 15:41:30.729586 44716 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0224 15:41:30.729655 44716 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0224 15:41:30.729727 44716 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0224 15:41:30.729770 44716 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0224 15:41:30.729815 44716 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0224 15:41:30.729861 44716 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0224 15:41:30.729926 44716 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0224 15:41:30.730019 44716 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0224 15:41:30.730062 44716 kubeadm.go:322] [certs] Using the existing "sa" key
I0224 15:41:30.730140 44716 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0224 15:41:30.766967 44716 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0224 15:41:31.076115 44716 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0224 15:41:31.384960 44716 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0224 15:41:31.516906 44716 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0224 15:41:31.517912 44716 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0224 15:41:31.539561 44716 out.go:204] - Booting up control plane ...
I0224 15:41:31.539729 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0224 15:41:31.539879 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0224 15:41:31.540011 44716 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0224 15:41:31.540121 44716 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0224 15:41:31.540380 44716 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0224 15:42:11.527410 44716 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0224 15:42:11.528560 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:42:11.528805 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:42:16.530407 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:42:16.530627 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:42:26.532359 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:42:26.532553 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:42:46.534480 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:42:46.534691 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:43:26.536979 44716 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0224 15:43:26.537207 44716 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0224 15:43:26.537220 44716 kubeadm.go:322]
I0224 15:43:26.537306 44716 kubeadm.go:322] Unfortunately, an error has occurred:
I0224 15:43:26.537361 44716 kubeadm.go:322] timed out waiting for the condition
I0224 15:43:26.537372 44716 kubeadm.go:322]
I0224 15:43:26.537413 44716 kubeadm.go:322] This error is likely caused by:
I0224 15:43:26.537447 44716 kubeadm.go:322] - The kubelet is not running
I0224 15:43:26.537565 44716 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0224 15:43:26.537576 44716 kubeadm.go:322]
I0224 15:43:26.537731 44716 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0224 15:43:26.537788 44716 kubeadm.go:322] - 'systemctl status kubelet'
I0224 15:43:26.537841 44716 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0224 15:43:26.537851 44716 kubeadm.go:322]
I0224 15:43:26.537957 44716 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0224 15:43:26.538053 44716 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0224 15:43:26.538144 44716 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0224 15:43:26.538209 44716 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0224 15:43:26.538312 44716 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0224 15:43:26.538352 44716 kubeadm.go:322] - 'docker logs CONTAINERID'
I0224 15:43:26.541202 44716 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0224 15:43:26.541287 44716 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0224 15:43:26.541378 44716 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
I0224 15:43:26.541451 44716 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0224 15:43:26.541527 44716 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0224 15:43:26.541592 44716 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0224 15:43:26.541638 44716 kubeadm.go:403] StartCluster complete in 3m53.602558278s
I0224 15:43:26.541732 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0224 15:43:26.560565 44716 logs.go:277] 0 containers: []
W0224 15:43:26.560579 44716 logs.go:279] No container was found matching "kube-apiserver"
I0224 15:43:26.560650 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0224 15:43:26.580329 44716 logs.go:277] 0 containers: []
W0224 15:43:26.580344 44716 logs.go:279] No container was found matching "etcd"
I0224 15:43:26.580415 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0224 15:43:26.599128 44716 logs.go:277] 0 containers: []
W0224 15:43:26.599142 44716 logs.go:279] No container was found matching "coredns"
I0224 15:43:26.599208 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0224 15:43:26.618127 44716 logs.go:277] 0 containers: []
W0224 15:43:26.618141 44716 logs.go:279] No container was found matching "kube-scheduler"
I0224 15:43:26.618222 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0224 15:43:26.636831 44716 logs.go:277] 0 containers: []
W0224 15:43:26.636844 44716 logs.go:279] No container was found matching "kube-proxy"
I0224 15:43:26.636916 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0224 15:43:26.656679 44716 logs.go:277] 0 containers: []
W0224 15:43:26.656692 44716 logs.go:279] No container was found matching "kube-controller-manager"
I0224 15:43:26.656760 44716 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0224 15:43:26.675681 44716 logs.go:277] 0 containers: []
W0224 15:43:26.675693 44716 logs.go:279] No container was found matching "kindnet"
I0224 15:43:26.675700 44716 logs.go:123] Gathering logs for kubelet ...
I0224 15:43:26.675710 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0224 15:43:26.714428 44716 logs.go:123] Gathering logs for dmesg ...
I0224 15:43:26.714446 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0224 15:43:26.728213 44716 logs.go:123] Gathering logs for describe nodes ...
I0224 15:43:26.728231 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0224 15:43:26.782923 44716 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0224 15:43:26.782936 44716 logs.go:123] Gathering logs for Docker ...
I0224 15:43:26.782943 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0224 15:43:26.807488 44716 logs.go:123] Gathering logs for container status ...
I0224 15:43:26.807503 44716 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0224 15:43:28.854675 44716 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.047141148s)
W0224 15:43:28.854822 44716 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0224 15:43:28.854846 44716 out.go:239] *
*
W0224 15:43:28.855038 44716 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0224 15:43:28.855075 44716 out.go:239] *
*
W0224 15:43:28.855750 44716 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0224 15:43:28.917693 44716 out.go:177]
W0224 15:43:28.959846 44716 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0224 15:43:28.960024 44716 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0224 15:43:28.960087 44716 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0224 15:43:29.001471 44716 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-583000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-583000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-583000:
-- stdout --
[
{
"Id": "9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa",
"Created": "2023-02-24T23:39:27.334833203Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 656762,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-02-24T23:39:27.64687431Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:b74f629d1852fc20b6085123d98944654faddf1d7e642b41aa2866d7a48081ea",
"ResolvConfPath": "/var/lib/docker/containers/9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa/hostname",
"HostsPath": "/var/lib/docker/containers/9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa/hosts",
"LogPath": "/var/lib/docker/containers/9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa/9cf258058c712ff0d19281911206a5b9dff66a4ac0958748a0ca2078b040fffa-json.log",
"Name": "/old-k8s-version-583000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-583000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-583000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/0378edae673977c276449a87574566f39b3e7178feb64aa8573e700d57017f07-init/diff:/var/lib/docker/overlay2/090bcc891b21f0d9b31c4a5b1a320c5e316289558e632a2b6c5e992d202fb20b/diff:/var/lib/docker/overlay2/d4038dc077274b7995ad7ae819cc855efc4eb5ece26de70285bfecf41ffeeaf0/diff:/var/lib/docker/overlay2/323d8b1f094a7a667b9ac22d681d79418dd180a1f48f4df11eb68204511a1900/diff:/var/lib/docker/overlay2/58c178b988b717cf5a66ca855d8eeafaeedbbecbb8d17b3c624da9d11da84298/diff:/var/lib/docker/overlay2/9cc1438791a51e9ed2a68dfdbebf6e6efe1b65ead288011935a3b80743da99b3/diff:/var/lib/docker/overlay2/57bff46aefba77057092b40bce71150843bf0396273131130e9807d1b4f49e64/diff:/var/lib/docker/overlay2/3e72f6db5681e2cc253f09aba7963765d8fd1533a1281f5bebd15cc32c6917eb/diff:/var/lib/docker/overlay2/6b9bf9d6e5a1d7e6bbba98ce5a8499b7d5783c593cc9748ea564e294b35db755/diff:/var/lib/docker/overlay2/ee777a22d8793697035ce27b4ea9217e5be9957b632769eaab350b54ab91ae9f/diff:/var/lib/docker/overlay2/e9e851
14730e02ef6096398efed46ec9a3917746bc11be0b94664e19be511d88/diff:/var/lib/docker/overlay2/5b998a77bcbd13fdcdb765faf9508458bc25089bd66cfda5b045dd1c00f81dde/diff:/var/lib/docker/overlay2/f9e79da15c2ae6e03de004791da8ab9e42fe890d60aee6de9f498a7ca5759c3b/diff:/var/lib/docker/overlay2/dc2cee5d7abc39c9318e6509d101802620afffde40cb40ce458a69eee2b596c7/diff:/var/lib/docker/overlay2/283655f6c15014eea16666a5dc2722a0affb0c63989379b417ffc172d79ab74e/diff:/var/lib/docker/overlay2/c123625f83f65d73fbae79bdcc7e7720e3afc660bd2bef55a0fcf5e6b0eed020/diff:/var/lib/docker/overlay2/e05ad0591b705fb7ebf2dbd767ed0d71b991218a02ddff77025837a5baab2181/diff:/var/lib/docker/overlay2/8324b7189c7dc594588028d0a241da19320051b98cf2eecbfb8e535dbe4828fd/diff:/var/lib/docker/overlay2/64d81e83771f64d5b5da39a8cba2980760fe27693c4e8f6be12bb75f8d7ee52a/diff:/var/lib/docker/overlay2/517cd4ce3dafee340299da3fac09378c877fc3becfa3f04eebc4ada282356790/diff:/var/lib/docker/overlay2/57e275726317c089e6a0ab3805ebdd3e762884089dc1dc3105b684b35c4f531e/diff:/var/lib/d
ocker/overlay2/861ff6c5ede4a603bbf860ae177b03a94994bef411ecf75e7c872ec757135fdc/diff:/var/lib/docker/overlay2/232c9bcdc5f0db2998256d21ae7ba58ca75e1dd4a4241797a4ea487263518a6a/diff:/var/lib/docker/overlay2/8d295896dcb1e09f3db53ef49b025df21f6e78b7ebb79f65a5fa27168702e8f0/diff:/var/lib/docker/overlay2/25ae29e984510ea508a8961c42d9a34730690423007d852d55a86191b21be913/diff:/var/lib/docker/overlay2/1a4879fcf6445660156e579037a1d9e6e2ddb1724d86797fa3ed3a1008d52950/diff:/var/lib/docker/overlay2/7b5fe07a6af400157cd971cd3fcd205488943ef28bed0f4306f1384221d0014b/diff:/var/lib/docker/overlay2/066cc6a66f3ee4d02cab8d3abd0f45f3c22fcdfa77adc5a82d8e526dbf3de5c0/diff:/var/lib/docker/overlay2/6c6c18c3d0df0e0add96377a7dfedf0513565ea145aa99ee672dc6b903d1a77d/diff:/var/lib/docker/overlay2/6861357e8485926b15888713d18c560aaa7d42d1278899a902d66234091e8a9d/diff:/var/lib/docker/overlay2/649cf18532e44c7c06d4480b7b048a072e4332bb7a1705ba0bcd418dd38ce0b9/diff:/var/lib/docker/overlay2/e655d5600a6a88950aa7ec0f38af04d2bda578ac23dd44970e8fb13703d
d08b7/diff:/var/lib/docker/overlay2/9c716cb8f3100de3f4cdbea2f4d7af8fa502516b6135097a1709296469f181e5/diff:/var/lib/docker/overlay2/18dac52ec743664ef1a9ca7c093035ab25db9c736fcec32e8b38d8db4434157e/diff:/var/lib/docker/overlay2/87e6a678acd66787264fe25f8e2fd1840ef476f6b7d969f16168694d101afe0b/diff:/var/lib/docker/overlay2/dca590590d1fb513a0d455929601ee877c0b9b6c247d06f1d11f83e871595f79/diff:/var/lib/docker/overlay2/157a7e690ef104d198b536028ab39be1c7b357a428d4ce8278aeb79f0ee74c0e/diff:/var/lib/docker/overlay2/a6f94e786d222ddcfdbfc14e1aa38a83b44430727fcafa2a47968395f2594111/diff:/var/lib/docker/overlay2/f3f6a39c3e36660974945c281b08bd20e12e4a85414f7716da020ee3afb53c9f/diff:/var/lib/docker/overlay2/71cd12298612d61c3229a50fa5c0359db0057e9a96e39ad20cdb8ea48dbfe559/diff:/var/lib/docker/overlay2/8c8ae2ac0512cc12aede073c1eeaae85b89c880d2bd544194b94e3a68db3fc07/diff:/var/lib/docker/overlay2/00e5186e2ada52ba9bb84c56be91327d3cebfb4f918de062856ecdc663a06f8a/diff:/var/lib/docker/overlay2/cbca231439ca50214d25e75010827f686ff701
248205654e17439440b9b11fe9/diff",
"MergedDir": "/var/lib/docker/overlay2/0378edae673977c276449a87574566f39b3e7178feb64aa8573e700d57017f07/merged",
"UpperDir": "/var/lib/docker/overlay2/0378edae673977c276449a87574566f39b3e7178feb64aa8573e700d57017f07/diff",
"WorkDir": "/var/lib/docker/overlay2/0378edae673977c276449a87574566f39b3e7178feb64aa8573e700d57017f07/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-583000",
"Source": "/var/lib/docker/volumes/old-k8s-version-583000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-583000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1676506612-15768@sha256:cc1cb283879fedae93096946a6953a50075ed680d467a47cbf669e0ed7d3aebc",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-583000",
"name.minikube.sigs.k8s.io": "old-k8s-version-583000",
"org.opencontainers.image.ref.name": "ubuntu",
"org.opencontainers.image.version": "20.04",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "ccbbf2decf41649f41df74885d1d793fc99a23332ed7cbdace844e27ae8d2f97",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61528"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61529"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61530"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61526"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "61527"
}
]
},
"SandboxKey": "/var/run/docker/netns/ccbbf2decf41",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-583000": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"9cf258058c71",
"old-k8s-version-583000"
],
"NetworkID": "2385c01da446b4577899b64cfb7c6c9559167bd3e5ad2b3a0a47d05890f119a8",
"EndpointID": "3462dd1dea9f8bb290104351b25276e0a1c016889fe2607476bc6b38a54327d3",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-583000 -n old-k8s-version-583000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-583000 -n old-k8s-version-583000: exit status 6 (398.301515ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0224 15:43:29.526945 45881 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-583000" does not appear in /Users/jenkins/minikube-integration/15909-26406/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-583000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (250.81s)