=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-867000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
E0128 11:14:04.171334 3849 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/addons-869000/client.crt: no such file or directory
E0128 11:14:16.356699 3849 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/calico-360000/client.crt: no such file or directory
E0128 11:14:16.587680 3849 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/skaffold-449000/client.crt: no such file or directory
=== CONT TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-867000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m11.145332396s)
-- stdout --
* [old-k8s-version-867000] minikube v1.29.0 on Darwin 13.2
- MINIKUBE_LOCATION=15565
- KUBECONFIG=/Users/jenkins/minikube-integration/15565-2556/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15565-2556/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-867000 in cluster old-k8s-version-867000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 20.10.23 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0128 11:13:53.156240 20834 out.go:296] Setting OutFile to fd 1 ...
I0128 11:13:53.156508 20834 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0128 11:13:53.156513 20834 out.go:309] Setting ErrFile to fd 2...
I0128 11:13:53.156517 20834 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0128 11:13:53.156627 20834 root.go:334] Updating PATH: /Users/jenkins/minikube-integration/15565-2556/.minikube/bin
I0128 11:13:53.157211 20834 out.go:303] Setting JSON to false
I0128 11:13:53.176401 20834 start.go:125] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":4408,"bootTime":1674928825,"procs":411,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.2","kernelVersion":"22.3.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}
W0128 11:13:53.176511 20834 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0128 11:13:53.199232 20834 out.go:177] * [old-k8s-version-867000] minikube v1.29.0 on Darwin 13.2
I0128 11:13:53.241653 20834 notify.go:220] Checking for updates...
I0128 11:13:53.263861 20834 out.go:177] - MINIKUBE_LOCATION=15565
I0128 11:13:53.285772 20834 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15565-2556/kubeconfig
I0128 11:13:53.307801 20834 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0128 11:13:53.336014 20834 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0128 11:13:53.356354 20834 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15565-2556/.minikube
I0128 11:13:53.398416 20834 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0128 11:13:53.420200 20834 config.go:180] Loaded profile config "kubenet-360000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0128 11:13:53.420318 20834 driver.go:365] Setting default libvirt URI to qemu:///system
I0128 11:13:53.481788 20834 docker.go:141] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0128 11:13:53.481948 20834 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0128 11:13:53.627027 20834 info.go:266] docker info: {ID:ZQRK:VGKA:BF77:XHQV:PEXM:RJZJ:SOX5:CSWG:2I44:5P6X:JP3N:A4CH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:60 OomKillDisable:false NGoroutines:57 SystemTime:2023-01-28 19:13:53.538397088 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0128 11:13:53.648644 20834 out.go:177] * Using the docker driver based on user configuration
I0128 11:13:53.669418 20834 start.go:296] selected driver: docker
I0128 11:13:53.669447 20834 start.go:857] validating driver "docker" against <nil>
I0128 11:13:53.669469 20834 start.go:868] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0128 11:13:53.673356 20834 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0128 11:13:53.821034 20834 info.go:266] docker info: {ID:ZQRK:VGKA:BF77:XHQV:PEXM:RJZJ:SOX5:CSWG:2I44:5P6X:JP3N:A4CH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:60 OomKillDisable:false NGoroutines:57 SystemTime:2023-01-28 19:13:53.729819194 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0128 11:13:53.821139 20834 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0128 11:13:53.821343 20834 start_flags.go:917] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0128 11:13:53.842619 20834 out.go:177] * Using Docker Desktop driver with root privileges
I0128 11:13:53.863363 20834 cni.go:84] Creating CNI manager for ""
I0128 11:13:53.863378 20834 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0128 11:13:53.863389 20834 start_flags.go:319] config:
{Name:old-k8s-version-867000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-867000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocke
t: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0128 11:13:53.905495 20834 out.go:177] * Starting control plane node old-k8s-version-867000 in cluster old-k8s-version-867000
I0128 11:13:53.926391 20834 cache.go:120] Beginning downloading kic base image for docker with docker
I0128 11:13:53.947503 20834 out.go:177] * Pulling base image ...
I0128 11:13:53.989533 20834 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0128 11:13:53.989569 20834 image.go:77] Checking for gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 in local docker daemon
I0128 11:13:53.989606 20834 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15565-2556/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0128 11:13:53.989620 20834 cache.go:57] Caching tarball of preloaded images
I0128 11:13:53.989746 20834 preload.go:174] Found /Users/jenkins/minikube-integration/15565-2556/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0128 11:13:53.989757 20834 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0128 11:13:53.990293 20834 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/config.json ...
I0128 11:13:53.990361 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/config.json: {Name:mk0cbf33f8de2c9f57b7efd946066a0212a1a31c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:13:54.050487 20834 image.go:81] Found gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 in local docker daemon, skipping pull
I0128 11:13:54.050504 20834 cache.go:143] gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 exists in daemon, skipping load
I0128 11:13:54.050523 20834 cache.go:193] Successfully downloaded all kic artifacts
I0128 11:13:54.050597 20834 start.go:364] acquiring machines lock for old-k8s-version-867000: {Name:mk6bff3692844ef15630a267932d689c213153ab Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0128 11:13:54.050759 20834 start.go:368] acquired machines lock for "old-k8s-version-867000" in 149.212µs
I0128 11:13:54.050793 20834 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-867000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-867000 Namespace:default APIServerName:minikubeCA APIServe
rNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fals
e CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0128 11:13:54.050875 20834 start.go:125] createHost starting for "" (driver="docker")
I0128 11:13:54.073952 20834 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0128 11:13:54.074233 20834 start.go:159] libmachine.API.Create for "old-k8s-version-867000" (driver="docker")
I0128 11:13:54.074263 20834 client.go:168] LocalClient.Create starting
I0128 11:13:54.074385 20834 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem
I0128 11:13:54.074432 20834 main.go:141] libmachine: Decoding PEM data...
I0128 11:13:54.074451 20834 main.go:141] libmachine: Parsing certificate...
I0128 11:13:54.074516 20834 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/cert.pem
I0128 11:13:54.074558 20834 main.go:141] libmachine: Decoding PEM data...
I0128 11:13:54.074568 20834 main.go:141] libmachine: Parsing certificate...
I0128 11:13:54.075020 20834 cli_runner.go:164] Run: docker network inspect old-k8s-version-867000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0128 11:13:54.134343 20834 cli_runner.go:211] docker network inspect old-k8s-version-867000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0128 11:13:54.134461 20834 network_create.go:281] running [docker network inspect old-k8s-version-867000] to gather additional debugging logs...
I0128 11:13:54.134475 20834 cli_runner.go:164] Run: docker network inspect old-k8s-version-867000
W0128 11:13:54.193899 20834 cli_runner.go:211] docker network inspect old-k8s-version-867000 returned with exit code 1
I0128 11:13:54.193930 20834 network_create.go:284] error running [docker network inspect old-k8s-version-867000]: docker network inspect old-k8s-version-867000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-867000
I0128 11:13:54.193943 20834 network_create.go:286] output of [docker network inspect old-k8s-version-867000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-867000
** /stderr **
I0128 11:13:54.194034 20834 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0128 11:13:54.254862 20834 network.go:209] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0128 11:13:54.256297 20834 network.go:209] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0128 11:13:54.257630 20834 network.go:209] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0128 11:13:54.257977 20834 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00120a100}
I0128 11:13:54.257988 20834 network_create.go:123] attempt to create docker network old-k8s-version-867000 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0128 11:13:54.258056 20834 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-867000 old-k8s-version-867000
I0128 11:13:54.354570 20834 network_create.go:107] docker network old-k8s-version-867000 192.168.76.0/24 created
I0128 11:13:54.354602 20834 kic.go:117] calculated static IP "192.168.76.2" for the "old-k8s-version-867000" container
I0128 11:13:54.354732 20834 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0128 11:13:54.416783 20834 cli_runner.go:164] Run: docker volume create old-k8s-version-867000 --label name.minikube.sigs.k8s.io=old-k8s-version-867000 --label created_by.minikube.sigs.k8s.io=true
I0128 11:13:54.475098 20834 oci.go:103] Successfully created a docker volume old-k8s-version-867000
I0128 11:13:54.475234 20834 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-867000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-867000 --entrypoint /usr/bin/test -v old-k8s-version-867000:/var gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 -d /var/lib
I0128 11:13:54.910288 20834 oci.go:107] Successfully prepared a docker volume old-k8s-version-867000
I0128 11:13:54.910322 20834 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0128 11:13:54.910354 20834 kic.go:190] Starting extracting preloaded images to volume ...
I0128 11:13:54.910456 20834 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15565-2556/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-867000:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 -I lz4 -xf /preloaded.tar -C /extractDir
I0128 11:14:00.721462 20834 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15565-2556/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-867000:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 -I lz4 -xf /preloaded.tar -C /extractDir: (5.811000944s)
I0128 11:14:00.721483 20834 kic.go:199] duration metric: took 5.811199 seconds to extract preloaded images to volume
I0128 11:14:00.721603 20834 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0128 11:14:00.876572 20834 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-867000 --name old-k8s-version-867000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-867000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-867000 --network old-k8s-version-867000 --ip 192.168.76.2 --volume old-k8s-version-867000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15
I0128 11:14:01.272739 20834 cli_runner.go:164] Run: docker container inspect old-k8s-version-867000 --format={{.State.Running}}
I0128 11:14:01.339510 20834 cli_runner.go:164] Run: docker container inspect old-k8s-version-867000 --format={{.State.Status}}
I0128 11:14:01.405818 20834 cli_runner.go:164] Run: docker exec old-k8s-version-867000 stat /var/lib/dpkg/alternatives/iptables
I0128 11:14:01.529813 20834 oci.go:144] the created container "old-k8s-version-867000" has a running status.
I0128 11:14:01.529847 20834 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa...
I0128 11:14:01.590519 20834 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0128 11:14:01.714767 20834 cli_runner.go:164] Run: docker container inspect old-k8s-version-867000 --format={{.State.Status}}
I0128 11:14:01.787479 20834 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0128 11:14:01.787502 20834 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-867000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0128 11:14:01.915379 20834 cli_runner.go:164] Run: docker container inspect old-k8s-version-867000 --format={{.State.Status}}
I0128 11:14:01.978168 20834 machine.go:88] provisioning docker machine ...
I0128 11:14:01.978218 20834 ubuntu.go:169] provisioning hostname "old-k8s-version-867000"
I0128 11:14:01.978306 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:02.038099 20834 main.go:141] libmachine: Using SSH client type: native
I0128 11:14:02.038368 20834 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55052 <nil> <nil>}
I0128 11:14:02.038383 20834 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-867000 && echo "old-k8s-version-867000" | sudo tee /etc/hostname
I0128 11:14:02.183215 20834 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-867000
I0128 11:14:02.183310 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:02.243802 20834 main.go:141] libmachine: Using SSH client type: native
I0128 11:14:02.243956 20834 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55052 <nil> <nil>}
I0128 11:14:02.243970 20834 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-867000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-867000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-867000' | sudo tee -a /etc/hosts;
fi
fi
I0128 11:14:02.376699 20834 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0128 11:14:02.376721 20834 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15565-2556/.minikube CaCertPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15565-2556/.minikube}
I0128 11:14:02.376740 20834 ubuntu.go:177] setting up certificates
I0128 11:14:02.376747 20834 provision.go:83] configureAuth start
I0128 11:14:02.376820 20834 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-867000
I0128 11:14:02.435951 20834 provision.go:138] copyHostCerts
I0128 11:14:02.436046 20834 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.pem, removing ...
I0128 11:14:02.436055 20834 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.pem
I0128 11:14:02.436170 20834 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.pem (1082 bytes)
I0128 11:14:02.436360 20834 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-2556/.minikube/cert.pem, removing ...
I0128 11:14:02.436366 20834 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-2556/.minikube/cert.pem
I0128 11:14:02.436431 20834 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15565-2556/.minikube/cert.pem (1123 bytes)
I0128 11:14:02.436570 20834 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-2556/.minikube/key.pem, removing ...
I0128 11:14:02.436576 20834 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-2556/.minikube/key.pem
I0128 11:14:02.436637 20834 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15565-2556/.minikube/key.pem (1679 bytes)
I0128 11:14:02.436747 20834 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-867000 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-867000]
I0128 11:14:02.584909 20834 provision.go:172] copyRemoteCerts
I0128 11:14:02.584969 20834 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0128 11:14:02.585019 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:02.648291 20834 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55052 SSHKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa Username:docker}
I0128 11:14:02.746763 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0128 11:14:02.765511 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0128 11:14:02.783947 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0128 11:14:02.802947 20834 provision.go:86] duration metric: configureAuth took 426.19243ms
I0128 11:14:02.802960 20834 ubuntu.go:193] setting minikube options for container-runtime
I0128 11:14:02.803101 20834 config.go:180] Loaded profile config "old-k8s-version-867000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0128 11:14:02.803160 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:02.864502 20834 main.go:141] libmachine: Using SSH client type: native
I0128 11:14:02.864691 20834 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55052 <nil> <nil>}
I0128 11:14:02.864708 20834 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0128 11:14:02.999595 20834 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0128 11:14:02.999615 20834 ubuntu.go:71] root file system type: overlay
I0128 11:14:02.999767 20834 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0128 11:14:02.999854 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:03.065015 20834 main.go:141] libmachine: Using SSH client type: native
I0128 11:14:03.065181 20834 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55052 <nil> <nil>}
I0128 11:14:03.065236 20834 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0128 11:14:03.210456 20834 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0128 11:14:03.210565 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:03.274129 20834 main.go:141] libmachine: Using SSH client type: native
I0128 11:14:03.274330 20834 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55052 <nil> <nil>}
I0128 11:14:03.274345 20834 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0128 11:14:03.899408 20834 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-01-19 17:34:14.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-01-28 19:14:03.207679914 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0128 11:14:03.899429 20834 machine.go:91] provisioned docker machine in 1.921251975s
I0128 11:14:03.899435 20834 client.go:171] LocalClient.Create took 9.825261606s
I0128 11:14:03.899455 20834 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-867000" took 9.82531427s
I0128 11:14:03.899463 20834 start.go:300] post-start starting for "old-k8s-version-867000" (driver="docker")
I0128 11:14:03.899476 20834 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0128 11:14:03.899580 20834 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0128 11:14:03.899660 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:03.962847 20834 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55052 SSHKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa Username:docker}
I0128 11:14:04.061713 20834 ssh_runner.go:195] Run: cat /etc/os-release
I0128 11:14:04.065856 20834 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0128 11:14:04.065873 20834 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0128 11:14:04.065882 20834 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0128 11:14:04.065887 20834 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0128 11:14:04.065900 20834 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15565-2556/.minikube/addons for local assets ...
I0128 11:14:04.066007 20834 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15565-2556/.minikube/files for local assets ...
I0128 11:14:04.066185 20834 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15565-2556/.minikube/files/etc/ssl/certs/38492.pem -> 38492.pem in /etc/ssl/certs
I0128 11:14:04.066411 20834 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0128 11:14:04.073990 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/files/etc/ssl/certs/38492.pem --> /etc/ssl/certs/38492.pem (1708 bytes)
I0128 11:14:04.092194 20834 start.go:303] post-start completed in 192.717674ms
I0128 11:14:04.092765 20834 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-867000
I0128 11:14:04.150482 20834 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/config.json ...
I0128 11:14:04.150980 20834 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0128 11:14:04.151058 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:04.211538 20834 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55052 SSHKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa Username:docker}
I0128 11:14:04.302470 20834 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0128 11:14:04.307608 20834 start.go:128] duration metric: createHost completed in 10.256822625s
I0128 11:14:04.307624 20834 start.go:83] releasing machines lock for "old-k8s-version-867000", held for 10.256953503s
I0128 11:14:04.307713 20834 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-867000
I0128 11:14:04.367007 20834 ssh_runner.go:195] Run: cat /version.json
I0128 11:14:04.367007 20834 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0128 11:14:04.367082 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:04.367136 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:04.433384 20834 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55052 SSHKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa Username:docker}
I0128 11:14:04.433378 20834 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55052 SSHKeyPath:/Users/jenkins/minikube-integration/15565-2556/.minikube/machines/old-k8s-version-867000/id_rsa Username:docker}
I0128 11:14:04.739360 20834 ssh_runner.go:195] Run: systemctl --version
I0128 11:14:04.744868 20834 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0128 11:14:04.750604 20834 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0128 11:14:04.774533 20834 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0128 11:14:04.774614 20834 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0128 11:14:04.790405 20834 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0128 11:14:04.799629 20834 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0128 11:14:04.799647 20834 start.go:483] detecting cgroup driver to use...
I0128 11:14:04.799661 20834 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0128 11:14:04.799794 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0128 11:14:04.815026 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0128 11:14:04.825253 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0128 11:14:04.834577 20834 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0128 11:14:04.834648 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0128 11:14:04.849408 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0128 11:14:04.863985 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0128 11:14:04.875188 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0128 11:14:04.885612 20834 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0128 11:14:04.896077 20834 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0128 11:14:04.905181 20834 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0128 11:14:04.913644 20834 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0128 11:14:04.920835 20834 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0128 11:14:05.012624 20834 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0128 11:14:05.094583 20834 start.go:483] detecting cgroup driver to use...
I0128 11:14:05.094603 20834 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0128 11:14:05.094672 20834 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0128 11:14:05.105600 20834 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0128 11:14:05.105665 20834 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0128 11:14:05.117256 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0128 11:14:05.132180 20834 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0128 11:14:05.213255 20834 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0128 11:14:05.283940 20834 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0128 11:14:05.283956 20834 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0128 11:14:05.298397 20834 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0128 11:14:05.385232 20834 ssh_runner.go:195] Run: sudo systemctl restart docker
I0128 11:14:05.606246 20834 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0128 11:14:05.638924 20834 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0128 11:14:05.713739 20834 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 20.10.23 ...
I0128 11:14:05.713904 20834 cli_runner.go:164] Run: docker exec -t old-k8s-version-867000 dig +short host.docker.internal
I0128 11:14:05.839508 20834 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0128 11:14:05.839681 20834 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0128 11:14:05.844379 20834 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0128 11:14:05.854842 20834 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-867000
I0128 11:14:05.915009 20834 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0128 11:14:05.915091 20834 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0128 11:14:05.941481 20834 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0128 11:14:05.941505 20834 docker.go:560] Images already preloaded, skipping extraction
I0128 11:14:05.941599 20834 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0128 11:14:05.966423 20834 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0128 11:14:05.966442 20834 cache_images.go:84] Images are preloaded, skipping loading
I0128 11:14:05.966542 20834 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0128 11:14:06.040803 20834 cni.go:84] Creating CNI manager for ""
I0128 11:14:06.040824 20834 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0128 11:14:06.040845 20834 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0128 11:14:06.040860 20834 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-867000 NodeName:old-k8s-version-867000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0128 11:14:06.041063 20834 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-867000"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-867000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0128 11:14:06.041195 20834 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-867000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-867000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0128 11:14:06.041339 20834 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0128 11:14:06.049930 20834 binaries.go:44] Found k8s binaries, skipping transfer
I0128 11:14:06.049999 20834 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0128 11:14:06.057706 20834 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0128 11:14:06.072007 20834 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0128 11:14:06.087299 20834 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0128 11:14:06.101055 20834 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0128 11:14:06.105161 20834 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0128 11:14:06.115368 20834 certs.go:56] Setting up /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000 for IP: 192.168.76.2
I0128 11:14:06.115404 20834 certs.go:186] acquiring lock for shared ca certs: {Name:mkee0a6d4b79657122da9b64494daa75cd779ac0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.115646 20834 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.key
I0128 11:14:06.115720 20834 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15565-2556/.minikube/proxy-client-ca.key
I0128 11:14:06.115764 20834 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.key
I0128 11:14:06.115781 20834 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.crt with IP's: []
I0128 11:14:06.228928 20834 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.crt ...
I0128 11:14:06.228943 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.crt: {Name:mkb3fdc745b9dc6f0f289131b70e0f27743cca81 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.229285 20834 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.key ...
I0128 11:14:06.229294 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/client.key: {Name:mkb0b27cfa70a612df49dfaa86a068a07bd23a84 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.229579 20834 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key.31bdca25
I0128 11:14:06.229599 20834 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0128 11:14:06.310379 20834 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt.31bdca25 ...
I0128 11:14:06.310402 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt.31bdca25: {Name:mkbfb8c17515f917d29094d995f99fdde8375832 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.310715 20834 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key.31bdca25 ...
I0128 11:14:06.310724 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key.31bdca25: {Name:mk89f76483b098a70ccfe60227b4d25aa76b4380 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.310911 20834 certs.go:333] copying /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt
I0128 11:14:06.311080 20834 certs.go:337] copying /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key
I0128 11:14:06.311291 20834 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.key
I0128 11:14:06.311307 20834 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.crt with IP's: []
I0128 11:14:06.426795 20834 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.crt ...
I0128 11:14:06.426810 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.crt: {Name:mk1149fc8926c3f2be0e15d27dd713ec1295f563 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.427094 20834 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.key ...
I0128 11:14:06.427107 20834 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.key: {Name:mkbe4b1512e2656cf3334323a963863947836d8e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0128 11:14:06.427515 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/3849.pem (1338 bytes)
W0128 11:14:06.427570 20834 certs.go:397] ignoring /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/3849_empty.pem, impossibly tiny 0 bytes
I0128 11:14:06.427581 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca-key.pem (1679 bytes)
I0128 11:14:06.427613 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/ca.pem (1082 bytes)
I0128 11:14:06.427643 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/cert.pem (1123 bytes)
I0128 11:14:06.427676 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/certs/key.pem (1679 bytes)
I0128 11:14:06.427751 20834 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-2556/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15565-2556/.minikube/files/etc/ssl/certs/38492.pem (1708 bytes)
I0128 11:14:06.428339 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0128 11:14:06.447711 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0128 11:14:06.466032 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0128 11:14:06.484971 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/profiles/old-k8s-version-867000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0128 11:14:06.502878 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0128 11:14:06.521072 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0128 11:14:06.539560 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0128 11:14:06.557885 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0128 11:14:06.575884 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/files/etc/ssl/certs/38492.pem --> /usr/share/ca-certificates/38492.pem (1708 bytes)
I0128 11:14:06.593890 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0128 11:14:06.611988 20834 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-2556/.minikube/certs/3849.pem --> /usr/share/ca-certificates/3849.pem (1338 bytes)
I0128 11:14:06.630936 20834 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0128 11:14:06.644239 20834 ssh_runner.go:195] Run: openssl version
I0128 11:14:06.650021 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/38492.pem && ln -fs /usr/share/ca-certificates/38492.pem /etc/ssl/certs/38492.pem"
I0128 11:14:06.658423 20834 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/38492.pem
I0128 11:14:06.663018 20834 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Jan 28 18:26 /usr/share/ca-certificates/38492.pem
I0128 11:14:06.663070 20834 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/38492.pem
I0128 11:14:06.668657 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/38492.pem /etc/ssl/certs/3ec20f2e.0"
I0128 11:14:06.677437 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0128 11:14:06.685769 20834 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0128 11:14:06.689985 20834 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Jan 28 18:22 /usr/share/ca-certificates/minikubeCA.pem
I0128 11:14:06.690054 20834 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0128 11:14:06.695762 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0128 11:14:06.704009 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3849.pem && ln -fs /usr/share/ca-certificates/3849.pem /etc/ssl/certs/3849.pem"
I0128 11:14:06.712602 20834 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3849.pem
I0128 11:14:06.716843 20834 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Jan 28 18:26 /usr/share/ca-certificates/3849.pem
I0128 11:14:06.716893 20834 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3849.pem
I0128 11:14:06.722465 20834 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/3849.pem /etc/ssl/certs/51391683.0"
I0128 11:14:06.730956 20834 kubeadm.go:401] StartCluster: {Name:old-k8s-version-867000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-867000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs
:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQe
muFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0128 11:14:06.731117 20834 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0128 11:14:06.754808 20834 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0128 11:14:06.763862 20834 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0128 11:14:06.772447 20834 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0128 11:14:06.772510 20834 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0128 11:14:06.781327 20834 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0128 11:14:06.781355 20834 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0128 11:14:06.835722 20834 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0128 11:14:06.835764 20834 kubeadm.go:322] [preflight] Running pre-flight checks
I0128 11:14:07.139094 20834 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0128 11:14:07.139221 20834 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0128 11:14:07.139328 20834 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0128 11:14:07.379107 20834 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0128 11:14:07.379911 20834 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0128 11:14:07.386182 20834 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0128 11:14:07.464472 20834 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0128 11:14:07.488944 20834 out.go:204] - Generating certificates and keys ...
I0128 11:14:07.489012 20834 kubeadm.go:322] [certs] Using existing ca certificate authority
I0128 11:14:07.489087 20834 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0128 11:14:07.658417 20834 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0128 11:14:07.782655 20834 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0128 11:14:07.886308 20834 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0128 11:14:08.120743 20834 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0128 11:14:08.281837 20834 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0128 11:14:08.281946 20834 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0128 11:14:08.422000 20834 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0128 11:14:08.422142 20834 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0128 11:14:08.530181 20834 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0128 11:14:08.647033 20834 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0128 11:14:08.776931 20834 kubeadm.go:322] [certs] Generating "sa" key and public key
I0128 11:14:08.777019 20834 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0128 11:14:08.844247 20834 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0128 11:14:09.043866 20834 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0128 11:14:09.122644 20834 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0128 11:14:09.298572 20834 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0128 11:14:09.299109 20834 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0128 11:14:09.320937 20834 out.go:204] - Booting up control plane ...
I0128 11:14:09.321032 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0128 11:14:09.321119 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0128 11:14:09.321177 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0128 11:14:09.321254 20834 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0128 11:14:09.321390 20834 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0128 11:14:49.308182 20834 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0128 11:14:49.308819 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:14:49.308970 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:14:54.309732 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:14:54.309895 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:15:04.311267 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:15:04.311486 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:15:24.313095 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:15:24.313353 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:16:04.314362 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:16:04.314605 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:16:04.314627 20834 kubeadm.go:322]
I0128 11:16:04.314674 20834 kubeadm.go:322] Unfortunately, an error has occurred:
I0128 11:16:04.314721 20834 kubeadm.go:322] timed out waiting for the condition
I0128 11:16:04.314733 20834 kubeadm.go:322]
I0128 11:16:04.314778 20834 kubeadm.go:322] This error is likely caused by:
I0128 11:16:04.314827 20834 kubeadm.go:322] - The kubelet is not running
I0128 11:16:04.314970 20834 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0128 11:16:04.314987 20834 kubeadm.go:322]
I0128 11:16:04.315085 20834 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0128 11:16:04.315122 20834 kubeadm.go:322] - 'systemctl status kubelet'
I0128 11:16:04.315151 20834 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0128 11:16:04.315156 20834 kubeadm.go:322]
I0128 11:16:04.315277 20834 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0128 11:16:04.315384 20834 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0128 11:16:04.315479 20834 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0128 11:16:04.315537 20834 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0128 11:16:04.315614 20834 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0128 11:16:04.315667 20834 kubeadm.go:322] - 'docker logs CONTAINERID'
I0128 11:16:04.318134 20834 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0128 11:16:04.318205 20834 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0128 11:16:04.318304 20834 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
I0128 11:16:04.318386 20834 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0128 11:16:04.318465 20834 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0128 11:16:04.318531 20834 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0128 11:16:04.318710 20834 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-867000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0128 11:16:04.318740 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0128 11:16:04.738232 20834 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0128 11:16:04.751282 20834 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0128 11:16:04.751367 20834 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0128 11:16:04.760445 20834 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0128 11:16:04.760472 20834 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0128 11:16:04.810670 20834 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0128 11:16:04.810733 20834 kubeadm.go:322] [preflight] Running pre-flight checks
I0128 11:16:05.118458 20834 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0128 11:16:05.118555 20834 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0128 11:16:05.118637 20834 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0128 11:16:05.348374 20834 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0128 11:16:05.349090 20834 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0128 11:16:05.355789 20834 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0128 11:16:05.418751 20834 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0128 11:16:05.442967 20834 out.go:204] - Generating certificates and keys ...
I0128 11:16:05.443085 20834 kubeadm.go:322] [certs] Using existing ca certificate authority
I0128 11:16:05.443149 20834 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0128 11:16:05.443216 20834 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0128 11:16:05.443266 20834 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0128 11:16:05.443318 20834 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0128 11:16:05.443359 20834 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0128 11:16:05.443414 20834 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0128 11:16:05.443471 20834 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0128 11:16:05.443521 20834 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0128 11:16:05.443588 20834 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0128 11:16:05.443625 20834 kubeadm.go:322] [certs] Using the existing "sa" key
I0128 11:16:05.443666 20834 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0128 11:16:05.888483 20834 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0128 11:16:06.303949 20834 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0128 11:16:06.410562 20834 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0128 11:16:06.639237 20834 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0128 11:16:06.639894 20834 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0128 11:16:06.683170 20834 out.go:204] - Booting up control plane ...
I0128 11:16:06.683291 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0128 11:16:06.683377 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0128 11:16:06.683470 20834 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0128 11:16:06.683551 20834 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0128 11:16:06.683701 20834 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0128 11:16:46.650523 20834 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0128 11:16:46.651332 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:16:46.651550 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:16:51.651976 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:16:51.652143 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:17:01.652951 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:17:01.653104 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:17:21.653436 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:17:21.653626 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:18:01.654715 20834 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0128 11:18:01.654923 20834 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0128 11:18:01.654934 20834 kubeadm.go:322]
I0128 11:18:01.655001 20834 kubeadm.go:322] Unfortunately, an error has occurred:
I0128 11:18:01.655064 20834 kubeadm.go:322] timed out waiting for the condition
I0128 11:18:01.655073 20834 kubeadm.go:322]
I0128 11:18:01.655131 20834 kubeadm.go:322] This error is likely caused by:
I0128 11:18:01.655174 20834 kubeadm.go:322] - The kubelet is not running
I0128 11:18:01.655283 20834 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0128 11:18:01.655292 20834 kubeadm.go:322]
I0128 11:18:01.655393 20834 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0128 11:18:01.655432 20834 kubeadm.go:322] - 'systemctl status kubelet'
I0128 11:18:01.655467 20834 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0128 11:18:01.655476 20834 kubeadm.go:322]
I0128 11:18:01.655599 20834 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0128 11:18:01.655716 20834 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0128 11:18:01.655810 20834 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0128 11:18:01.655893 20834 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0128 11:18:01.656012 20834 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0128 11:18:01.656075 20834 kubeadm.go:322] - 'docker logs CONTAINERID'
I0128 11:18:01.658202 20834 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0128 11:18:01.658277 20834 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0128 11:18:01.658388 20834 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
I0128 11:18:01.658481 20834 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0128 11:18:01.658552 20834 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0128 11:18:01.658599 20834 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0128 11:18:01.658638 20834 kubeadm.go:403] StartCluster complete in 3m54.929924407s
I0128 11:18:01.658726 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0128 11:18:01.682347 20834 logs.go:279] 0 containers: []
W0128 11:18:01.682361 20834 logs.go:281] No container was found matching "kube-apiserver"
I0128 11:18:01.682454 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0128 11:18:01.706128 20834 logs.go:279] 0 containers: []
W0128 11:18:01.706141 20834 logs.go:281] No container was found matching "etcd"
I0128 11:18:01.706210 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0128 11:18:01.728908 20834 logs.go:279] 0 containers: []
W0128 11:18:01.728921 20834 logs.go:281] No container was found matching "coredns"
I0128 11:18:01.728996 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0128 11:18:01.753167 20834 logs.go:279] 0 containers: []
W0128 11:18:01.753181 20834 logs.go:281] No container was found matching "kube-scheduler"
I0128 11:18:01.753250 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0128 11:18:01.775979 20834 logs.go:279] 0 containers: []
W0128 11:18:01.775992 20834 logs.go:281] No container was found matching "kube-proxy"
I0128 11:18:01.776061 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0128 11:18:01.799423 20834 logs.go:279] 0 containers: []
W0128 11:18:01.799438 20834 logs.go:281] No container was found matching "kubernetes-dashboard"
I0128 11:18:01.799523 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0128 11:18:01.823708 20834 logs.go:279] 0 containers: []
W0128 11:18:01.823722 20834 logs.go:281] No container was found matching "storage-provisioner"
I0128 11:18:01.823794 20834 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0128 11:18:01.847374 20834 logs.go:279] 0 containers: []
W0128 11:18:01.847386 20834 logs.go:281] No container was found matching "kube-controller-manager"
I0128 11:18:01.847393 20834 logs.go:124] Gathering logs for kubelet ...
I0128 11:18:01.847403 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0128 11:18:01.886235 20834 logs.go:124] Gathering logs for dmesg ...
I0128 11:18:01.886254 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0128 11:18:01.900408 20834 logs.go:124] Gathering logs for describe nodes ...
I0128 11:18:01.900422 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0128 11:18:01.955303 20834 logs.go:131] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0128 11:18:01.955314 20834 logs.go:124] Gathering logs for Docker ...
I0128 11:18:01.955320 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0128 11:18:01.972921 20834 logs.go:124] Gathering logs for container status ...
I0128 11:18:01.972935 20834 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0128 11:18:04.025251 20834 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.052323622s)
W0128 11:18:04.025386 20834 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0128 11:18:04.025401 20834 out.go:239] *
*
W0128 11:18:04.025521 20834 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0128 11:18:04.025534 20834 out.go:239] *
*
W0128 11:18:04.026182 20834 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0128 11:18:04.110508 20834 out.go:177]
W0128 11:18:04.153069 20834 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0128 11:18:04.153179 20834 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0128 11:18:04.153217 20834 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0128 11:18:04.174646 20834 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-867000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-867000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-867000:
-- stdout --
[
{
"Id": "6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4",
"Created": "2023-01-28T19:14:00.935880886Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 284607,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-01-28T19:14:01.264450163Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:01c0ce65fff70ab1f019aa14679c46b23331bd108ae899438e589673efaa9c00",
"ResolvConfPath": "/var/lib/docker/containers/6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4/hostname",
"HostsPath": "/var/lib/docker/containers/6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4/hosts",
"LogPath": "/var/lib/docker/containers/6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4/6fd303c5a4731b02b4c33f7ed54a03d5f05077fd2b6a0c5bd12077806d5484b4-json.log",
"Name": "/old-k8s-version-867000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-867000:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-867000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/6248d9ed8b6aecbed69ddb29d60b03a9ba849c6a97ad9d5e99f1e4969e7a34cb-init/diff:/var/lib/docker/overlay2/79142c1bfb7158d99171fa66335a26cb13f2a8a8cbfa638c237bffed361e3a89/diff:/var/lib/docker/overlay2/e0e1fdc3516530c880b802c7996a1b8ce82ca95934105a41f254ec5137fd39e2/diff:/var/lib/docker/overlay2/fac4ba40ee55baf2806d43764f680a3decaf4fd8b368bbaea39adb65c5622ca5/diff:/var/lib/docker/overlay2/e441c934bc48afc9e7a3386f7c74fe0c20393e198bcc7743e8f9afdf05efe326/diff:/var/lib/docker/overlay2/d39bd1a2e948812990ca711385ce5fa40c2dc4804530a386d99c80b7f7bf6fe2/diff:/var/lib/docker/overlay2/95e11949c4f876ab44bd0adcbe6781a6adf47c3ff9e63ec97fd322466164eb6d/diff:/var/lib/docker/overlay2/63d3d3a1f2065070879db8c5bfb59a21af9a85c0fc71bc3330bd7cf269f4662b/diff:/var/lib/docker/overlay2/4d7e309fbeb00560ca7804415050f0c209f3d375cbbf236c28c11c70436d4ae7/diff:/var/lib/docker/overlay2/ba0d0617dbaa03778329a421f7fa9c42f1bb9e1e193a334dcd28b9dd83d388ed/diff:/var/lib/docker/overlay2/64bc6c
4c97d7afd10818bb2aee713912c62e6c6bad568397a334214568819094/diff:/var/lib/docker/overlay2/9eea8322dbca25f19e6b165fe69b4576c511d61932fa9488f54100b4adeda168/diff:/var/lib/docker/overlay2/ec78b4d745f797c0757e92219d722526d17cc9334aa98eb28fd654323898f059/diff:/var/lib/docker/overlay2/b63329cd62781660f2238fbcf704c8eebb0ea9c063e7692adfb1d54a5956b76a/diff:/var/lib/docker/overlay2/be4ad500dc73dc0f7a89386a220fda9a34cf83a2943e0df5f43e79bfeeec6dfb/diff:/var/lib/docker/overlay2/cc506fb9628569db47233dde2107f623c36f8706857dc9175ecc18da27f21ca9/diff:/var/lib/docker/overlay2/d3fbb137518a7e6371da37751ff1fb77c913000ef6751293d79279f527c805d0/diff:/var/lib/docker/overlay2/de9b2061ccfcc155f185f7ab9847b5efdcdc77c3dd2e26c7e010b4786b19466e/diff:/var/lib/docker/overlay2/47068d751b648d6786ed5645603f9500f2d3549961d067c28722a53af0072a33/diff:/var/lib/docker/overlay2/6404c0f71023a39e6175130d1bfc9a1f4d2eae9a418fb7e0d42c0a65317606c7/diff:/var/lib/docker/overlay2/bd3f3a98034631dd17e4c4d411d8babd82c3bf642410f52f8af6f71acbc09106/diff:/var/lib/d
ocker/overlay2/4e0a7618854eea772703e589408f79580161b9177c879421f2f691c46d58a60a/diff:/var/lib/docker/overlay2/782fb02ecc3c1bc71373ff3d8b713b2bc4d26a60de3da9576878ade33b4992ee/diff:/var/lib/docker/overlay2/7533e761a436c07c8d9cd30e8b859b1f85de596787d3e4f00ba2fc87c8e08809/diff:/var/lib/docker/overlay2/8fa41de6ca6cee76164e50650a0b671b453322b8cada6868d2090bdc55dca493/diff:/var/lib/docker/overlay2/dcac84778124f3f93c0704b8ce7a776f24b386bba206afb9fa8657f6361de17b/diff:/var/lib/docker/overlay2/38476836b7aea22bb21f8df4c5d24ca581ec51456577cbc587735fd7632f83ec/diff:/var/lib/docker/overlay2/b180f265391afb4bbd508de68ada783469c21c620f1796782ffb3b573f7e70a2/diff:/var/lib/docker/overlay2/e13f4fcd119c410ddd745414d8b1d0ae30714a3cdbe36d7b01819005d41464aa/diff:/var/lib/docker/overlay2/690e7538a41741ca2ccf5aeec1133ccbc188dc6cc1dce00935059a30f6cb0c9b/diff:/var/lib/docker/overlay2/1765a1cbadca6aa0cdaaba760dedeba82919d483a8ad99943e888f737518b687/diff:/var/lib/docker/overlay2/2d7069c458db8901c6e152ca71b0aaa1ddb0a3457c7c8fb7bb040671d2b
a42ae/diff:/var/lib/docker/overlay2/7e4848df7b6b74fc7d6c4c0fc99075bdb69362e7527b6f677e7d2124d02cecd1/diff:/var/lib/docker/overlay2/c6645f05d6483a2e5e109899c766fee254537cb272ed8b25f40da02dec68bd0a/diff:/var/lib/docker/overlay2/eec788e4d45314574efe5c16f7502c0f5a09febe1c8ee35a5180259889f8257f/diff:/var/lib/docker/overlay2/45cd4b08a855f084c1c06a65f871df9287fe4fa5492eb93ea8c5806f8902af34/diff:/var/lib/docker/overlay2/bc8f511ffbc35a69047b9052add80532a88f0a305785aa0ffecee72babecdb6c/diff:/var/lib/docker/overlay2/72b0909462bee1f7a5f130f21715b150d3ed694f6d1f8f94bebc3b882ffd37b4/diff:/var/lib/docker/overlay2/8989993d4ea98ef674ee8268e3df0a227279d8ecd9c6cc96bde872992753da1f/diff:/var/lib/docker/overlay2/f914250e3f8befc8b24c98ac5561328b3df75d319ed91a9d1efe4287edf819ed/diff:/var/lib/docker/overlay2/00034316e473aca001ab0dceff5d356002633ffac50bc9df58da1c6c6bd9dc1b/diff:/var/lib/docker/overlay2/c321f77609367af7b9b056846695b79a6ca7011dae1346ccb7b268424d848661/diff:/var/lib/docker/overlay2/791cadd07a627ebff13560c239469308a2ad30
659ca32e469a18745c54fcc7fe/diff:/var/lib/docker/overlay2/67a4def3de9e3f2fe0bf3da0abe7b7679ee2a173be572e7ebdc5bab7db1c321b/diff:/var/lib/docker/overlay2/9f1255e61d7efdef3846a0ec873eb647e15ce7d8183aacccf1e9790726dbebcd/diff",
"MergedDir": "/var/lib/docker/overlay2/6248d9ed8b6aecbed69ddb29d60b03a9ba849c6a97ad9d5e99f1e4969e7a34cb/merged",
"UpperDir": "/var/lib/docker/overlay2/6248d9ed8b6aecbed69ddb29d60b03a9ba849c6a97ad9d5e99f1e4969e7a34cb/diff",
"WorkDir": "/var/lib/docker/overlay2/6248d9ed8b6aecbed69ddb29d60b03a9ba849c6a97ad9d5e99f1e4969e7a34cb/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-867000",
"Source": "/var/lib/docker/volumes/old-k8s-version-867000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-867000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.37@sha256:8bf7a0e8a062bc5e2b71d28b35bfa9cc862d9220e234e86176b3785f685d8b15",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-867000",
"name.minikube.sigs.k8s.io": "old-k8s-version-867000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "1965b08fee08b8af3afcb0cd99ff5e9095d1796192376cf6c580470e41c37ec4",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55052"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55053"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55054"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55055"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55056"
}
]
},
"SandboxKey": "/var/run/docker/netns/1965b08fee08",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-867000": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"6fd303c5a473",
"old-k8s-version-867000"
],
"NetworkID": "05da8fabe29d00d6e3eb58e11e2bbe3932ea7f3d437268a555d06945d4a9c8c9",
"EndpointID": "135b64967c344fdb2ea21fbb73e05567bd790c79731b1337b03704ac6cb97d2a",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-867000 -n old-k8s-version-867000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-867000 -n old-k8s-version-867000: exit status 6 (424.627232ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0128 11:18:04.749541 21903 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-867000" does not appear in /Users/jenkins/minikube-integration/15565-2556/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-867000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (251.67s)