=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-115000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
E0124 10:28:36.550325 4355 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/enable-default-cni-129000/client.crt: no such file or directory
=== CONT TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-115000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m12.140509612s)
-- stdout --
* [old-k8s-version-115000] minikube v1.28.0 on Darwin 13.1
- MINIKUBE_LOCATION=15565
- KUBECONFIG=/Users/jenkins/minikube-integration/15565-3057/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15565-3057/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-115000 in cluster old-k8s-version-115000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 20.10.22 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0124 10:28:31.509172 25956 out.go:296] Setting OutFile to fd 1 ...
I0124 10:28:31.509331 25956 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0124 10:28:31.509336 25956 out.go:309] Setting ErrFile to fd 2...
I0124 10:28:31.509342 25956 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0124 10:28:31.509454 25956 root.go:334] Updating PATH: /Users/jenkins/minikube-integration/15565-3057/.minikube/bin
I0124 10:28:31.509986 25956 out.go:303] Setting JSON to false
I0124 10:28:31.528751 25956 start.go:125] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":5286,"bootTime":1674579625,"procs":440,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.1","kernelVersion":"22.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}
W0124 10:28:31.528848 25956 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0124 10:28:31.549505 25956 out.go:177] * [old-k8s-version-115000] minikube v1.28.0 on Darwin 13.1
I0124 10:28:31.592208 25956 notify.go:220] Checking for updates...
I0124 10:28:31.613226 25956 out.go:177] - MINIKUBE_LOCATION=15565
I0124 10:28:31.672190 25956 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15565-3057/kubeconfig
I0124 10:28:31.729964 25956 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0124 10:28:31.805404 25956 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0124 10:28:31.865265 25956 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15565-3057/.minikube
I0124 10:28:31.924057 25956 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0124 10:28:31.961811 25956 config.go:180] Loaded profile config "false-129000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0124 10:28:31.961898 25956 driver.go:365] Setting default libvirt URI to qemu:///system
I0124 10:28:32.029001 25956 docker.go:141] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0124 10:28:32.029148 25956 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0124 10:28:32.187098 25956 info.go:266] docker info: {ID:CJUQ:42TS:YYE6:SCGN:MW5S:IHFM:RBEN:XGTW:DFJE:IUNN:DVU7:AORP Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:61 OomKillDisable:false NGoroutines:56 SystemTime:2023-01-24 18:28:32.085461027 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0124 10:28:32.230104 25956 out.go:177] * Using the docker driver based on user configuration
I0124 10:28:32.251051 25956 start.go:296] selected driver: docker
I0124 10:28:32.251067 25956 start.go:840] validating driver "docker" against <nil>
I0124 10:28:32.251080 25956 start.go:851] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0124 10:28:32.253997 25956 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0124 10:28:32.399433 25956 info.go:266] docker info: {ID:CJUQ:42TS:YYE6:SCGN:MW5S:IHFM:RBEN:XGTW:DFJE:IUNN:DVU7:AORP Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:61 OomKillDisable:false NGoroutines:56 SystemTime:2023-01-24 18:28:32.30354817 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe
rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default
name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path:
/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0124 10:28:32.399574 25956 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0124 10:28:32.399713 25956 start_flags.go:917] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0124 10:28:32.422662 25956 out.go:177] * Using Docker Desktop driver with root privileges
I0124 10:28:32.444135 25956 cni.go:84] Creating CNI manager for ""
I0124 10:28:32.444176 25956 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0124 10:28:32.444194 25956 start_flags.go:319] config:
{Name:old-k8s-version-115000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-115000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:}
I0124 10:28:32.487341 25956 out.go:177] * Starting control plane node old-k8s-version-115000 in cluster old-k8s-version-115000
I0124 10:28:32.508498 25956 cache.go:120] Beginning downloading kic base image for docker with docker
I0124 10:28:32.531521 25956 out.go:177] * Pulling base image ...
I0124 10:28:32.574302 25956 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0124 10:28:32.574308 25956 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a in local docker daemon
I0124 10:28:32.574357 25956 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15565-3057/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0124 10:28:32.574372 25956 cache.go:57] Caching tarball of preloaded images
I0124 10:28:32.574520 25956 preload.go:174] Found /Users/jenkins/minikube-integration/15565-3057/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0124 10:28:32.574530 25956 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0124 10:28:32.574969 25956 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/config.json ...
I0124 10:28:32.575117 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/config.json: {Name:mk1b224550e635a9178b3a0ce48d666e9f326f54 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:32.633751 25956 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a in local docker daemon, skipping pull
I0124 10:28:32.633778 25956 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a exists in daemon, skipping load
I0124 10:28:32.633795 25956 cache.go:193] Successfully downloaded all kic artifacts
I0124 10:28:32.633857 25956 start.go:364] acquiring machines lock for old-k8s-version-115000: {Name:mk8bd7ad2f5bf8d8f939782c15c7e824af20d268 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0124 10:28:32.634017 25956 start.go:368] acquired machines lock for "old-k8s-version-115000" in 147.544µs
I0124 10:28:32.634045 25956 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-115000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-115000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0124 10:28:32.634098 25956 start.go:125] createHost starting for "" (driver="docker")
I0124 10:28:32.656719 25956 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0124 10:28:32.657120 25956 start.go:159] libmachine.API.Create for "old-k8s-version-115000" (driver="docker")
I0124 10:28:32.657155 25956 client.go:168] LocalClient.Create starting
I0124 10:28:32.657311 25956 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem
I0124 10:28:32.657388 25956 main.go:141] libmachine: Decoding PEM data...
I0124 10:28:32.657416 25956 main.go:141] libmachine: Parsing certificate...
I0124 10:28:32.657510 25956 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/cert.pem
I0124 10:28:32.657571 25956 main.go:141] libmachine: Decoding PEM data...
I0124 10:28:32.657587 25956 main.go:141] libmachine: Parsing certificate...
I0124 10:28:32.685230 25956 cli_runner.go:164] Run: docker network inspect old-k8s-version-115000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0124 10:28:32.741830 25956 cli_runner.go:211] docker network inspect old-k8s-version-115000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0124 10:28:32.741934 25956 network_create.go:281] running [docker network inspect old-k8s-version-115000] to gather additional debugging logs...
I0124 10:28:32.741954 25956 cli_runner.go:164] Run: docker network inspect old-k8s-version-115000
W0124 10:28:32.798322 25956 cli_runner.go:211] docker network inspect old-k8s-version-115000 returned with exit code 1
I0124 10:28:32.798354 25956 network_create.go:284] error running [docker network inspect old-k8s-version-115000]: docker network inspect old-k8s-version-115000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-115000
I0124 10:28:32.798369 25956 network_create.go:286] output of [docker network inspect old-k8s-version-115000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-115000
** /stderr **
I0124 10:28:32.798447 25956 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0124 10:28:32.855534 25956 network.go:209] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0124 10:28:32.857110 25956 network.go:209] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0124 10:28:32.858651 25956 network.go:209] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0124 10:28:32.858940 25956 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0012892c0}
I0124 10:28:32.858950 25956 network_create.go:123] attempt to create docker network old-k8s-version-115000 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0124 10:28:32.859022 25956 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-115000 old-k8s-version-115000
I0124 10:28:32.947074 25956 network_create.go:107] docker network old-k8s-version-115000 192.168.76.0/24 created
I0124 10:28:32.947111 25956 kic.go:117] calculated static IP "192.168.76.2" for the "old-k8s-version-115000" container
I0124 10:28:32.947234 25956 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0124 10:28:33.004979 25956 cli_runner.go:164] Run: docker volume create old-k8s-version-115000 --label name.minikube.sigs.k8s.io=old-k8s-version-115000 --label created_by.minikube.sigs.k8s.io=true
I0124 10:28:33.060648 25956 oci.go:103] Successfully created a docker volume old-k8s-version-115000
I0124 10:28:33.060783 25956 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-115000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-115000 --entrypoint /usr/bin/test -v old-k8s-version-115000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a -d /var/lib
I0124 10:28:33.536323 25956 oci.go:107] Successfully prepared a docker volume old-k8s-version-115000
I0124 10:28:33.536380 25956 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0124 10:28:33.536394 25956 kic.go:190] Starting extracting preloaded images to volume ...
I0124 10:28:33.536516 25956 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15565-3057/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-115000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a -I lz4 -xf /preloaded.tar -C /extractDir
I0124 10:28:39.474451 25956 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15565-3057/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-115000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a -I lz4 -xf /preloaded.tar -C /extractDir: (5.937818048s)
I0124 10:28:39.474475 25956 kic.go:199] duration metric: took 5.938043 seconds to extract preloaded images to volume
I0124 10:28:39.474608 25956 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0124 10:28:39.626184 25956 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-115000 --name old-k8s-version-115000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-115000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-115000 --network old-k8s-version-115000 --ip 192.168.76.2 --volume old-k8s-version-115000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a
I0124 10:28:40.037774 25956 cli_runner.go:164] Run: docker container inspect old-k8s-version-115000 --format={{.State.Running}}
I0124 10:28:40.113116 25956 cli_runner.go:164] Run: docker container inspect old-k8s-version-115000 --format={{.State.Status}}
I0124 10:28:40.188282 25956 cli_runner.go:164] Run: docker exec old-k8s-version-115000 stat /var/lib/dpkg/alternatives/iptables
I0124 10:28:40.349124 25956 oci.go:144] the created container "old-k8s-version-115000" has a running status.
I0124 10:28:40.349156 25956 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa...
I0124 10:28:40.464599 25956 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0124 10:28:40.588354 25956 cli_runner.go:164] Run: docker container inspect old-k8s-version-115000 --format={{.State.Status}}
I0124 10:28:40.651908 25956 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0124 10:28:40.651926 25956 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-115000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0124 10:28:40.764895 25956 cli_runner.go:164] Run: docker container inspect old-k8s-version-115000 --format={{.State.Status}}
I0124 10:28:40.822944 25956 machine.go:88] provisioning docker machine ...
I0124 10:28:40.822986 25956 ubuntu.go:169] provisioning hostname "old-k8s-version-115000"
I0124 10:28:40.823082 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:40.915598 25956 main.go:141] libmachine: Using SSH client type: native
I0124 10:28:40.915795 25956 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55234 <nil> <nil>}
I0124 10:28:40.915811 25956 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-115000 && echo "old-k8s-version-115000" | sudo tee /etc/hostname
I0124 10:28:41.056408 25956 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-115000
I0124 10:28:41.056530 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:41.116723 25956 main.go:141] libmachine: Using SSH client type: native
I0124 10:28:41.116883 25956 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55234 <nil> <nil>}
I0124 10:28:41.116896 25956 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-115000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-115000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-115000' | sudo tee -a /etc/hosts;
fi
fi
I0124 10:28:41.250887 25956 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0124 10:28:41.250907 25956 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15565-3057/.minikube CaCertPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15565-3057/.minikube}
I0124 10:28:41.250923 25956 ubuntu.go:177] setting up certificates
I0124 10:28:41.250931 25956 provision.go:83] configureAuth start
I0124 10:28:41.251003 25956 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-115000
I0124 10:28:41.309764 25956 provision.go:138] copyHostCerts
I0124 10:28:41.309860 25956 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-3057/.minikube/cert.pem, removing ...
I0124 10:28:41.309867 25956 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-3057/.minikube/cert.pem
I0124 10:28:41.309996 25956 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15565-3057/.minikube/cert.pem (1123 bytes)
I0124 10:28:41.310214 25956 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-3057/.minikube/key.pem, removing ...
I0124 10:28:41.310220 25956 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-3057/.minikube/key.pem
I0124 10:28:41.310282 25956 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15565-3057/.minikube/key.pem (1675 bytes)
I0124 10:28:41.310425 25956 exec_runner.go:144] found /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.pem, removing ...
I0124 10:28:41.310430 25956 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.pem
I0124 10:28:41.310490 25956 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.pem (1078 bytes)
I0124 10:28:41.310601 25956 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-115000 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-115000]
I0124 10:28:41.368230 25956 provision.go:172] copyRemoteCerts
I0124 10:28:41.368287 25956 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0124 10:28:41.368345 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:41.428090 25956 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55234 SSHKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa Username:docker}
I0124 10:28:41.523472 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0124 10:28:41.550928 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0124 10:28:41.567962 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0124 10:28:41.584879 25956 provision.go:86] duration metric: configureAuth took 333.93455ms
I0124 10:28:41.584892 25956 ubuntu.go:193] setting minikube options for container-runtime
I0124 10:28:41.585557 25956 config.go:180] Loaded profile config "old-k8s-version-115000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0124 10:28:41.585706 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:41.643781 25956 main.go:141] libmachine: Using SSH client type: native
I0124 10:28:41.643929 25956 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55234 <nil> <nil>}
I0124 10:28:41.643941 25956 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0124 10:28:41.778757 25956 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0124 10:28:41.778775 25956 ubuntu.go:71] root file system type: overlay
I0124 10:28:41.778923 25956 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0124 10:28:41.779003 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:41.837952 25956 main.go:141] libmachine: Using SSH client type: native
I0124 10:28:41.838119 25956 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55234 <nil> <nil>}
I0124 10:28:41.838174 25956 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0124 10:28:41.979942 25956 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0124 10:28:41.980068 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:42.037635 25956 main.go:141] libmachine: Using SSH client type: native
I0124 10:28:42.037806 25956 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55234 <nil> <nil>}
I0124 10:28:42.037819 25956 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0124 10:28:42.669952 25956 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2022-12-15 22:25:58.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-01-24 18:28:41.977119040 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0124 10:28:42.669970 25956 machine.go:91] provisioned docker machine in 1.84699497s
I0124 10:28:42.669977 25956 client.go:171] LocalClient.Create took 10.012747664s
I0124 10:28:42.669996 25956 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-115000" took 10.012810568s
I0124 10:28:42.670007 25956 start.go:300] post-start starting for "old-k8s-version-115000" (driver="docker")
I0124 10:28:42.670014 25956 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0124 10:28:42.670096 25956 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0124 10:28:42.670157 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:42.730379 25956 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55234 SSHKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa Username:docker}
I0124 10:28:42.823846 25956 ssh_runner.go:195] Run: cat /etc/os-release
I0124 10:28:42.827462 25956 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0124 10:28:42.827480 25956 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0124 10:28:42.827487 25956 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0124 10:28:42.827493 25956 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0124 10:28:42.827501 25956 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15565-3057/.minikube/addons for local assets ...
I0124 10:28:42.827595 25956 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15565-3057/.minikube/files for local assets ...
I0124 10:28:42.827760 25956 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15565-3057/.minikube/files/etc/ssl/certs/43552.pem -> 43552.pem in /etc/ssl/certs
I0124 10:28:42.827948 25956 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0124 10:28:42.835536 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/files/etc/ssl/certs/43552.pem --> /etc/ssl/certs/43552.pem (1708 bytes)
I0124 10:28:42.853181 25956 start.go:303] post-start completed in 183.159278ms
I0124 10:28:42.853729 25956 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-115000
I0124 10:28:42.918610 25956 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/config.json ...
I0124 10:28:42.919071 25956 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0124 10:28:42.919162 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:42.980373 25956 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55234 SSHKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa Username:docker}
I0124 10:28:43.071582 25956 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0124 10:28:43.076280 25956 start.go:128] duration metric: createHost completed in 10.442104059s
I0124 10:28:43.076304 25956 start.go:83] releasing machines lock for "old-k8s-version-115000", held for 10.442209882s
I0124 10:28:43.076396 25956 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-115000
I0124 10:28:43.134272 25956 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0124 10:28:43.134274 25956 ssh_runner.go:195] Run: cat /version.json
I0124 10:28:43.134368 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:43.134376 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:43.198090 25956 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55234 SSHKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa Username:docker}
I0124 10:28:43.198120 25956 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55234 SSHKeyPath:/Users/jenkins/minikube-integration/15565-3057/.minikube/machines/old-k8s-version-115000/id_rsa Username:docker}
I0124 10:28:43.290386 25956 ssh_runner.go:195] Run: systemctl --version
I0124 10:28:43.496295 25956 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0124 10:28:43.501423 25956 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0124 10:28:43.521138 25956 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0124 10:28:43.521215 25956 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0124 10:28:43.535188 25956 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0124 10:28:43.543087 25956 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0124 10:28:43.543108 25956 start.go:472] detecting cgroup driver to use...
I0124 10:28:43.543128 25956 detect.go:158] detected "cgroupfs" cgroup driver on host os
I0124 10:28:43.543249 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0124 10:28:43.556895 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0124 10:28:43.565678 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0124 10:28:43.573953 25956 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0124 10:28:43.574017 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0124 10:28:43.582941 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0124 10:28:43.591850 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0124 10:28:43.600992 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0124 10:28:43.610293 25956 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0124 10:28:43.618455 25956 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0124 10:28:43.627536 25956 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0124 10:28:43.634951 25956 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0124 10:28:43.642995 25956 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0124 10:28:43.722565 25956 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0124 10:28:43.804041 25956 start.go:472] detecting cgroup driver to use...
I0124 10:28:43.804075 25956 detect.go:158] detected "cgroupfs" cgroup driver on host os
I0124 10:28:43.804167 25956 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0124 10:28:43.818062 25956 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0124 10:28:43.818147 25956 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0124 10:28:43.836624 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0124 10:28:43.860493 25956 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0124 10:28:43.977880 25956 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0124 10:28:44.090534 25956 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0124 10:28:44.090557 25956 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0124 10:28:44.119401 25956 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0124 10:28:44.215890 25956 ssh_runner.go:195] Run: sudo systemctl restart docker
I0124 10:28:44.516323 25956 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0124 10:28:44.551647 25956 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0124 10:28:44.632845 25956 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 20.10.22 ...
I0124 10:28:44.632935 25956 cli_runner.go:164] Run: docker exec -t old-k8s-version-115000 dig +short host.docker.internal
I0124 10:28:44.751796 25956 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0124 10:28:44.751939 25956 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0124 10:28:44.757524 25956 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0124 10:28:44.773121 25956 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-115000
I0124 10:28:44.836559 25956 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0124 10:28:44.836636 25956 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0124 10:28:44.863178 25956 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0124 10:28:44.863197 25956 docker.go:560] Images already preloaded, skipping extraction
I0124 10:28:44.863272 25956 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0124 10:28:44.889038 25956 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0124 10:28:44.889055 25956 cache_images.go:84] Images are preloaded, skipping loading
I0124 10:28:44.889163 25956 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0124 10:28:44.964631 25956 cni.go:84] Creating CNI manager for ""
I0124 10:28:44.964647 25956 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0124 10:28:44.964665 25956 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0124 10:28:44.964679 25956 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-115000 NodeName:old-k8s-version-115000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0124 10:28:44.964816 25956 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-115000"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-115000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0124 10:28:44.964891 25956 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-115000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-115000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0124 10:28:44.964953 25956 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0124 10:28:44.973216 25956 binaries.go:44] Found k8s binaries, skipping transfer
I0124 10:28:44.973294 25956 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0124 10:28:44.980952 25956 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0124 10:28:44.994090 25956 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0124 10:28:45.007247 25956 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0124 10:28:45.020177 25956 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0124 10:28:45.024168 25956 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0124 10:28:45.034366 25956 certs.go:56] Setting up /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000 for IP: 192.168.76.2
I0124 10:28:45.034385 25956 certs.go:186] acquiring lock for shared ca certs: {Name:mk86da88dcf76a0f52f98f7208bc1d04a2e55c04 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.034558 25956 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.key
I0124 10:28:45.034621 25956 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15565-3057/.minikube/proxy-client-ca.key
I0124 10:28:45.034662 25956 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.key
I0124 10:28:45.034675 25956 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.crt with IP's: []
I0124 10:28:45.112842 25956 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.crt ...
I0124 10:28:45.112857 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.crt: {Name:mk9d3eee06bd6f4f151bbbaa90edb30e08bcbe5f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.113155 25956 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.key ...
I0124 10:28:45.113163 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/client.key: {Name:mkdbf9876ce64ae9c5554d1fb1033561547482d2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.113366 25956 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key.31bdca25
I0124 10:28:45.113381 25956 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0124 10:28:45.433293 25956 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt.31bdca25 ...
I0124 10:28:45.433316 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt.31bdca25: {Name:mk94fb23ee75d9b0b85c00eba728838de3dd0246 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.433625 25956 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key.31bdca25 ...
I0124 10:28:45.433634 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key.31bdca25: {Name:mkbcf57a010e8df6693d6223d5b3b8c6c32f2c82 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.433823 25956 certs.go:333] copying /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt
I0124 10:28:45.433986 25956 certs.go:337] copying /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key
I0124 10:28:45.434139 25956 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.key
I0124 10:28:45.434153 25956 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.crt with IP's: []
I0124 10:28:45.561859 25956 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.crt ...
I0124 10:28:45.561874 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.crt: {Name:mkd987f474d8fdeaddffca722d43ad9c1a12b0aa Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.562189 25956 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.key ...
I0124 10:28:45.562201 25956 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.key: {Name:mkf7b914a52218a7cad603a7dbf3ad9aa202ca32 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0124 10:28:45.562608 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/4355.pem (1338 bytes)
W0124 10:28:45.562657 25956 certs.go:397] ignoring /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/4355_empty.pem, impossibly tiny 0 bytes
I0124 10:28:45.562668 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca-key.pem (1679 bytes)
I0124 10:28:45.562701 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/ca.pem (1078 bytes)
I0124 10:28:45.562733 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/cert.pem (1123 bytes)
I0124 10:28:45.562765 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/certs/key.pem (1675 bytes)
I0124 10:28:45.562838 25956 certs.go:401] found cert: /Users/jenkins/minikube-integration/15565-3057/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15565-3057/.minikube/files/etc/ssl/certs/43552.pem (1708 bytes)
I0124 10:28:45.563364 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0124 10:28:45.585275 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0124 10:28:45.606440 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0124 10:28:45.629746 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/profiles/old-k8s-version-115000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0124 10:28:45.653638 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0124 10:28:45.678760 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0124 10:28:45.702054 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0124 10:28:45.723737 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0124 10:28:45.749172 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/files/etc/ssl/certs/43552.pem --> /usr/share/ca-certificates/43552.pem (1708 bytes)
I0124 10:28:45.769205 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0124 10:28:45.794492 25956 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15565-3057/.minikube/certs/4355.pem --> /usr/share/ca-certificates/4355.pem (1338 bytes)
I0124 10:28:45.816057 25956 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0124 10:28:45.829982 25956 ssh_runner.go:195] Run: openssl version
I0124 10:28:45.836811 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/43552.pem && ln -fs /usr/share/ca-certificates/43552.pem /etc/ssl/certs/43552.pem"
I0124 10:28:45.853498 25956 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/43552.pem
I0124 10:28:45.858237 25956 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Jan 24 17:34 /usr/share/ca-certificates/43552.pem
I0124 10:28:45.858308 25956 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/43552.pem
I0124 10:28:45.864193 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/43552.pem /etc/ssl/certs/3ec20f2e.0"
I0124 10:28:45.873849 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0124 10:28:45.883121 25956 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0124 10:28:45.887881 25956 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Jan 24 17:28 /usr/share/ca-certificates/minikubeCA.pem
I0124 10:28:45.887935 25956 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0124 10:28:45.893729 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0124 10:28:45.901996 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4355.pem && ln -fs /usr/share/ca-certificates/4355.pem /etc/ssl/certs/4355.pem"
I0124 10:28:45.910409 25956 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4355.pem
I0124 10:28:45.914442 25956 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Jan 24 17:34 /usr/share/ca-certificates/4355.pem
I0124 10:28:45.914487 25956 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4355.pem
I0124 10:28:45.919921 25956 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/4355.pem /etc/ssl/certs/51391683.0"
I0124 10:28:45.928883 25956 kubeadm.go:401] StartCluster: {Name:old-k8s-version-115000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-115000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:}
I0124 10:28:45.929018 25956 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0124 10:28:45.954657 25956 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0124 10:28:45.963391 25956 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0124 10:28:45.971993 25956 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0124 10:28:45.972065 25956 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0124 10:28:45.982105 25956 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0124 10:28:45.982146 25956 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0124 10:28:46.032785 25956 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0124 10:28:46.032856 25956 kubeadm.go:322] [preflight] Running pre-flight checks
I0124 10:28:46.364640 25956 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0124 10:28:46.364735 25956 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0124 10:28:46.364824 25956 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0124 10:28:46.602030 25956 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0124 10:28:46.603641 25956 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0124 10:28:46.610082 25956 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0124 10:28:46.675538 25956 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0124 10:28:46.695894 25956 out.go:204] - Generating certificates and keys ...
I0124 10:28:46.696092 25956 kubeadm.go:322] [certs] Using existing ca certificate authority
I0124 10:28:46.696207 25956 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0124 10:28:46.843199 25956 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0124 10:28:47.058611 25956 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0124 10:28:47.194647 25956 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0124 10:28:47.436314 25956 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0124 10:28:47.645334 25956 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0124 10:28:47.645514 25956 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0124 10:28:47.766564 25956 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0124 10:28:47.766756 25956 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0124 10:28:47.821720 25956 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0124 10:28:47.963353 25956 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0124 10:28:48.089871 25956 kubeadm.go:322] [certs] Generating "sa" key and public key
I0124 10:28:48.089940 25956 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0124 10:28:48.311646 25956 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0124 10:28:48.371473 25956 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0124 10:28:48.536180 25956 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0124 10:28:48.660898 25956 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0124 10:28:48.661999 25956 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0124 10:28:48.690175 25956 out.go:204] - Booting up control plane ...
I0124 10:28:48.690268 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0124 10:28:48.690333 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0124 10:28:48.690408 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0124 10:28:48.690478 25956 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0124 10:28:48.690595 25956 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0124 10:29:28.671429 25956 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0124 10:29:28.671571 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:29:28.671798 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:29:33.672153 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:29:33.672465 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:29:43.673747 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:29:43.673987 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:30:03.674404 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:30:03.674557 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:30:43.675757 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:30:43.676052 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:30:43.676065 25956 kubeadm.go:322]
I0124 10:30:43.676106 25956 kubeadm.go:322] Unfortunately, an error has occurred:
I0124 10:30:43.676144 25956 kubeadm.go:322] timed out waiting for the condition
I0124 10:30:43.676152 25956 kubeadm.go:322]
I0124 10:30:43.676191 25956 kubeadm.go:322] This error is likely caused by:
I0124 10:30:43.676233 25956 kubeadm.go:322] - The kubelet is not running
I0124 10:30:43.676343 25956 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0124 10:30:43.676356 25956 kubeadm.go:322]
I0124 10:30:43.676442 25956 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0124 10:30:43.676472 25956 kubeadm.go:322] - 'systemctl status kubelet'
I0124 10:30:43.676497 25956 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0124 10:30:43.676511 25956 kubeadm.go:322]
I0124 10:30:43.676631 25956 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0124 10:30:43.676749 25956 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0124 10:30:43.676841 25956 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0124 10:30:43.676896 25956 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0124 10:30:43.677011 25956 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0124 10:30:43.677083 25956 kubeadm.go:322] - 'docker logs CONTAINERID'
I0124 10:30:43.679698 25956 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0124 10:30:43.679767 25956 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0124 10:30:43.679883 25956 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
I0124 10:30:43.679993 25956 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0124 10:30:43.680088 25956 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0124 10:30:43.680153 25956 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0124 10:30:43.680337 25956 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-115000 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0124 10:30:43.680365 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0124 10:30:44.103272 25956 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0124 10:30:44.113371 25956 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0124 10:30:44.113434 25956 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0124 10:30:44.121269 25956 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0124 10:30:44.121294 25956 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0124 10:30:44.172739 25956 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0124 10:30:44.173988 25956 kubeadm.go:322] [preflight] Running pre-flight checks
I0124 10:30:44.514627 25956 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0124 10:30:44.514713 25956 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0124 10:30:44.514801 25956 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0124 10:30:44.760140 25956 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0124 10:30:44.762715 25956 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0124 10:30:44.770002 25956 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0124 10:30:44.841472 25956 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0124 10:30:44.862026 25956 out.go:204] - Generating certificates and keys ...
I0124 10:30:44.862093 25956 kubeadm.go:322] [certs] Using existing ca certificate authority
I0124 10:30:44.862155 25956 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0124 10:30:44.862231 25956 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0124 10:30:44.862287 25956 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0124 10:30:44.862347 25956 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0124 10:30:44.862397 25956 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0124 10:30:44.862453 25956 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0124 10:30:44.862502 25956 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0124 10:30:44.862557 25956 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0124 10:30:44.862615 25956 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0124 10:30:44.862646 25956 kubeadm.go:322] [certs] Using the existing "sa" key
I0124 10:30:44.862703 25956 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0124 10:30:45.013931 25956 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0124 10:30:45.324651 25956 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0124 10:30:45.584278 25956 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0124 10:30:45.985031 25956 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0124 10:30:45.985499 25956 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0124 10:30:46.021164 25956 out.go:204] - Booting up control plane ...
I0124 10:30:46.021275 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0124 10:30:46.021376 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0124 10:30:46.021428 25956 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0124 10:30:46.021487 25956 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0124 10:30:46.021619 25956 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0124 10:31:25.995033 25956 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0124 10:31:25.995566 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:31:25.995742 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:31:30.997508 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:31:30.997714 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:31:40.998319 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:31:40.998526 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:32:00.999849 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:32:01.000065 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:32:41.001697 25956 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0124 10:32:41.001932 25956 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0124 10:32:41.001946 25956 kubeadm.go:322]
I0124 10:32:41.001980 25956 kubeadm.go:322] Unfortunately, an error has occurred:
I0124 10:32:41.002050 25956 kubeadm.go:322] timed out waiting for the condition
I0124 10:32:41.002059 25956 kubeadm.go:322]
I0124 10:32:41.002100 25956 kubeadm.go:322] This error is likely caused by:
I0124 10:32:41.002142 25956 kubeadm.go:322] - The kubelet is not running
I0124 10:32:41.002257 25956 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0124 10:32:41.002269 25956 kubeadm.go:322]
I0124 10:32:41.002382 25956 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0124 10:32:41.002445 25956 kubeadm.go:322] - 'systemctl status kubelet'
I0124 10:32:41.002490 25956 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0124 10:32:41.002501 25956 kubeadm.go:322]
I0124 10:32:41.002650 25956 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0124 10:32:41.002763 25956 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0124 10:32:41.002870 25956 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0124 10:32:41.002924 25956 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0124 10:32:41.003018 25956 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0124 10:32:41.003056 25956 kubeadm.go:322] - 'docker logs CONTAINERID'
I0124 10:32:41.006009 25956 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0124 10:32:41.006077 25956 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0124 10:32:41.006181 25956 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
I0124 10:32:41.006280 25956 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0124 10:32:41.006355 25956 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0124 10:32:41.006416 25956 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0124 10:32:41.006446 25956 kubeadm.go:403] StartCluster complete in 3m55.07605061s
I0124 10:32:41.006530 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0124 10:32:41.029392 25956 logs.go:279] 0 containers: []
W0124 10:32:41.029406 25956 logs.go:281] No container was found matching "kube-apiserver"
I0124 10:32:41.029475 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0124 10:32:41.053530 25956 logs.go:279] 0 containers: []
W0124 10:32:41.053544 25956 logs.go:281] No container was found matching "etcd"
I0124 10:32:41.053626 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0124 10:32:41.076228 25956 logs.go:279] 0 containers: []
W0124 10:32:41.076240 25956 logs.go:281] No container was found matching "coredns"
I0124 10:32:41.076310 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0124 10:32:41.099860 25956 logs.go:279] 0 containers: []
W0124 10:32:41.099874 25956 logs.go:281] No container was found matching "kube-scheduler"
I0124 10:32:41.099952 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0124 10:32:41.122215 25956 logs.go:279] 0 containers: []
W0124 10:32:41.122228 25956 logs.go:281] No container was found matching "kube-proxy"
I0124 10:32:41.122297 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0124 10:32:41.145143 25956 logs.go:279] 0 containers: []
W0124 10:32:41.145159 25956 logs.go:281] No container was found matching "kubernetes-dashboard"
I0124 10:32:41.145226 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0124 10:32:41.169703 25956 logs.go:279] 0 containers: []
W0124 10:32:41.169717 25956 logs.go:281] No container was found matching "storage-provisioner"
I0124 10:32:41.169786 25956 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0124 10:32:41.194802 25956 logs.go:279] 0 containers: []
W0124 10:32:41.194815 25956 logs.go:281] No container was found matching "kube-controller-manager"
I0124 10:32:41.194831 25956 logs.go:124] Gathering logs for container status ...
I0124 10:32:41.194839 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0124 10:32:43.245535 25956 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.050668154s)
I0124 10:32:43.245690 25956 logs.go:124] Gathering logs for kubelet ...
I0124 10:32:43.245700 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0124 10:32:43.282200 25956 logs.go:124] Gathering logs for dmesg ...
I0124 10:32:43.282216 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0124 10:32:43.294630 25956 logs.go:124] Gathering logs for describe nodes ...
I0124 10:32:43.294644 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0124 10:32:43.351069 25956 logs.go:131] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0124 10:32:43.351082 25956 logs.go:124] Gathering logs for Docker ...
I0124 10:32:43.351089 25956 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
W0124 10:32:43.368000 25956 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0124 10:32:43.368021 25956 out.go:239] *
*
W0124 10:32:43.368135 25956 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0124 10:32:43.368152 25956 out.go:239] *
*
W0124 10:32:43.368877 25956 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0124 10:32:43.453717 25956 out.go:177]
W0124 10:32:43.496559 25956 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.22. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0124 10:32:43.496727 25956 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0124 10:32:43.496800 25956 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0124 10:32:43.518720 25956 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-115000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-115000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-115000:
-- stdout --
[
{
"Id": "a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7",
"Created": "2023-01-24T18:28:39.694404231Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 291454,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-01-24T18:28:40.028695555Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:c4f6061730f518104bba7f63d4b9eb2ccd1634c6b2943801ca33b3f1c3908566",
"ResolvConfPath": "/var/lib/docker/containers/a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7/hostname",
"HostsPath": "/var/lib/docker/containers/a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7/hosts",
"LogPath": "/var/lib/docker/containers/a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7/a86b483b846749f4d97fdbe2ec82d893df12b3f9c2715903c32559271be484d7-json.log",
"Name": "/old-k8s-version-115000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-115000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-115000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/8e4380f78d315a88c5e4323101f897e468d006d2b599f317c78710e9195396e0-init/diff:/var/lib/docker/overlay2/9c481b6fc4eda78a970c6b00c96c9f145b5d588195952c506fee1f3a38f6b827/diff:/var/lib/docker/overlay2/47481f9bc7214dddcf62375b5df8c94540c57703b1d6d51ac990043359419126/diff:/var/lib/docker/overlay2/b598a508e6cab858c4e3e0eb9d52d91575db756297302c95c476bb85082f44e0/diff:/var/lib/docker/overlay2/eb2938493863a763d06f90aa9a8774a6cf5891162d070c28c5ea356826cb9c00/diff:/var/lib/docker/overlay2/32de44262b0214eac920d9f248eab5c56907a2e7f6aa69c29c4e46c4074a7cac/diff:/var/lib/docker/overlay2/1a295a0be202f67a8806d1a09e94b6509e529ccee06e851bf52c63e932a41598/diff:/var/lib/docker/overlay2/f31de4b92cc5bd6d7f65b450b1907917e4402f07bc4105fb25aa1f8246c4a4e5/diff:/var/lib/docker/overlay2/91ed229d1177ed82e2ad15ef48cab26e269f4aac446f236ccf7602f7812b4844/diff:/var/lib/docker/overlay2/d76c7ee0e38e12ccde73d3892d573e4aace99b016da7a10649752a4fe4fe8638/diff:/var/lib/docker/overlay2/5ded14
3e1cfe20dc794363143043800e9ddaa724457e76dfc3480bc88bdcf50b/diff:/var/lib/docker/overlay2/93a2cbd1b1d5abd2ffd6d5d72e40d5e932e3cdee4ef9b08c0ff0e340f0b82250/diff:/var/lib/docker/overlay2/bb3f5e5d2796cb20f7b19463a04326cc5e79a70f606d384a018bf2677fdc9c59/diff:/var/lib/docker/overlay2/9cbefe5a0d987b7c33fa83edf8c1c1e50920e178002ea80eff9cdf15510e9f7b/diff:/var/lib/docker/overlay2/33e82fcf8ab49ebe12677ce9de48d21aef3b5793b44b0bcabc73c3674c581010/diff:/var/lib/docker/overlay2/e11a40d223ac259cc9385f5df423a4114d92d60488a253e2c2690c0b7457a8e8/diff:/var/lib/docker/overlay2/61a6833055db19e06daf765b9f85b5e3a0b5c194642afb06034ca3aba1f55909/diff:/var/lib/docker/overlay2/ee3319bed930e23040288482eca784537f46b47463ff427584d9b2b5211797e9/diff:/var/lib/docker/overlay2/eee58dbcea38f98302a06040b8570e985d703290fd458ac7329bfa0e55bd8448/diff:/var/lib/docker/overlay2/b658fe28fcf5e1fc3d6b2e9b4b847d6121a8b983e9c7fb1985d5ab2346e98f8b/diff:/var/lib/docker/overlay2/31ea9d8f709524c49028c58c0f8a05346bb654e8f78be840f557b2bedf65a54a/diff:/var/lib/d
ocker/overlay2/3cf9440c13010ba35d99312d9b15c642407767bf1e0fc45d05a2549c249afec7/diff:/var/lib/docker/overlay2/800878867e03c9e3e3b31fd2704a00aa2ae0e3986914887bf3d409b9550ff520/diff:/var/lib/docker/overlay2/581fb56aa8110438414995ed4c4a14b9912c75de1736b4b7d23e0b8fe720ecd9/diff:/var/lib/docker/overlay2/07b86cd484da6c8fb2f4bee904f870665d71591e8dc7be5efa53e1794b81d00f/diff:/var/lib/docker/overlay2/1079786f8be32b332c09a1406a8c3309f812525af6501891ff134bf591345a8d/diff:/var/lib/docker/overlay2/aaaabdfc565926946334338a8c5990344cee39d099a4b9f5151467564c8b476e/diff:/var/lib/docker/overlay2/f39918c58fc40801faea98e8de1249f865a73650e08eab314c9d152e6a3b34b5/diff:/var/lib/docker/overlay2/974fee46992fba128bb6ec5407ff459f67134f9674349df844ad7efddb1ce197/diff:/var/lib/docker/overlay2/da1fb4192fa1a6e35f4cad8340c23595b9197c7b307dc3acbddafef7c3eabc82/diff:/var/lib/docker/overlay2/3b77e137962b80628f381627a68f75ff468d7ccae4458d96fa0579ac771218fe/diff:/var/lib/docker/overlay2/3faa717465cabff3baa6c98c646cb65ad47d063b215430893bb7f045676
3a794/diff:/var/lib/docker/overlay2/9597f4dd91d0689567f5b93707e47b90e9d9ba308488dff4c828008698d40802/diff:/var/lib/docker/overlay2/04df968177dde5eecc89152d1a182fb9b925a8092117778b87497db1e350ce61/diff:/var/lib/docker/overlay2/466c44ae57218cc92c3105e1e0070db81834724a69f8a4417391186e96073aae/diff:/var/lib/docker/overlay2/ec2a2a479f00a246da5470537471969c82985b2cfb98a4e35c9ff415d2a73141/diff:/var/lib/docker/overlay2/bce991b71b3cfd9fbda43bee1a05c1f7f5d1182ddbcf8f503851bd5874171f2b/diff:/var/lib/docker/overlay2/fe2929ef3680f5c3a0bd3eb8595ea97b8e43bba3f4c05803abf70a6c92a68114/diff:/var/lib/docker/overlay2/656654dcbc6d86ba911abe2cc555591ab82c37f6960cd0dad3551f4bf80122ee/diff:/var/lib/docker/overlay2/0e7faeb1891565534bd5d77c8619c32e7b51715e75dc04640b4ef0c3bc72a6b8/diff:/var/lib/docker/overlay2/9f0ce2e9bbc988036f4207b6b8237ba1f52855f6ee97e629aa058d0adcb5e7c4/diff:/var/lib/docker/overlay2/2bd42f9be2610b8910f7b39f5ce1792b9f608e775e6d3b39b12fef011686b5bd/diff:/var/lib/docker/overlay2/79703f59d1d8ea0485fb2360a75307a714a167
fc401d5ce68a58a9a201ea4152/diff:/var/lib/docker/overlay2/e589647f3d60f11426627ec161f2bf8878a577bc3388bb26e206fc5182f4f83c/diff:/var/lib/docker/overlay2/059bdd38117e61965c55837c7fd45d7e2825b162ea4e4cab6a656a436a7bbed6/diff",
"MergedDir": "/var/lib/docker/overlay2/8e4380f78d315a88c5e4323101f897e468d006d2b599f317c78710e9195396e0/merged",
"UpperDir": "/var/lib/docker/overlay2/8e4380f78d315a88c5e4323101f897e468d006d2b599f317c78710e9195396e0/diff",
"WorkDir": "/var/lib/docker/overlay2/8e4380f78d315a88c5e4323101f897e468d006d2b599f317c78710e9195396e0/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-115000",
"Source": "/var/lib/docker/volumes/old-k8s-version-115000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-115000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1674164627-15541@sha256:0a2280301e955e0d3910d6e639e0b7341db1f4a25558521ac97b38c782c6189a",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-115000",
"name.minikube.sigs.k8s.io": "old-k8s-version-115000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "aa575f37259f15aad30f1e98a1cee32e0ee3d18a75fba4a7e1f83ceb54a22cc0",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55234"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55235"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55236"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55232"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55233"
}
]
},
"SandboxKey": "/var/run/docker/netns/aa575f37259f",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-115000": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"a86b483b8467",
"old-k8s-version-115000"
],
"NetworkID": "5ad39a0309903e0f7f41a6f2aca4e1831033f2fa0e547dc51ca28338a4ce6eed",
"EndpointID": "d1a1a9e88cb895f566dfbf35e730e63d180921fd3d8f1717d0eebc147c038084",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-115000 -n old-k8s-version-115000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-115000 -n old-k8s-version-115000: exit status 6 (419.923984ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0124 10:32:44.095550 27086 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-115000" does not appear in /Users/jenkins/minikube-integration/15565-3057/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-115000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (252.66s)