=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-030235 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
E0114 03:02:46.612382 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.617493 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.628397 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.650573 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.690727 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.771190 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:46.931325 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:47.251491 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:47.892244 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:49.174334 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:51.735990 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:56.857642 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
E0114 03:02:58.414158 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/skaffold-024201/client.crt: no such file or directory
E0114 03:03:07.097987 2728 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/cilium-024326/client.crt: no such file or directory
=== CONT TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-030235 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m12.220281524s)
-- stdout --
* [old-k8s-version-030235] minikube v1.28.0 on Darwin 13.0.1
- MINIKUBE_LOCATION=15642
- KUBECONFIG=/Users/jenkins/minikube-integration/15642-1559/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15642-1559/.minikube
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-030235 in cluster old-k8s-version-030235
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 20.10.21 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0114 03:02:35.165501 16696 out.go:296] Setting OutFile to fd 1 ...
I0114 03:02:35.165693 16696 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0114 03:02:35.165700 16696 out.go:309] Setting ErrFile to fd 2...
I0114 03:02:35.165704 16696 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0114 03:02:35.165818 16696 root.go:334] Updating PATH: /Users/jenkins/minikube-integration/15642-1559/.minikube/bin
I0114 03:02:35.166394 16696 out.go:303] Setting JSON to false
I0114 03:02:35.186253 16696 start.go:125] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":3729,"bootTime":1673690426,"procs":389,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.0.1","kernelVersion":"22.1.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W0114 03:02:35.186385 16696 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0114 03:02:35.208944 16696 out.go:177] * [old-k8s-version-030235] minikube v1.28.0 on Darwin 13.0.1
I0114 03:02:35.266880 16696 notify.go:220] Checking for updates...
I0114 03:02:35.290828 16696 out.go:177] - MINIKUBE_LOCATION=15642
I0114 03:02:35.349725 16696 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15642-1559/kubeconfig
I0114 03:02:35.408315 16696 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0114 03:02:35.451456 16696 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0114 03:02:35.525793 16696 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15642-1559/.minikube
I0114 03:02:35.548136 16696 config.go:180] Loaded profile config "kubenet-024325": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.25.3
I0114 03:02:35.548231 16696 driver.go:365] Setting default libvirt URI to qemu:///system
I0114 03:02:35.661849 16696 docker.go:138] docker version: linux-20.10.21
I0114 03:02:35.662022 16696 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0114 03:02:35.836461 16696 info.go:266] docker info: {ID:VF6S:3GIL:4JQH:LPDQ:6EC6:D32C:6RZ7:IA3N:LZ7R:3YN2:QUOM:SIJ5 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:58 OomKillDisable:false NGoroutines:52 SystemTime:2023-01-14 11:02:35.718711008 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:3 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.21 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:1c90a442489720eec95342e1789ee8a5e1b9536f Expected:1c90a442489720eec95342e1789ee8a5e1b9536f} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/local/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1] map[Name:compose Path:/usr/local/lib/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.12.2] map[Name:dev Path:/usr/local/lib/docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.3] map[Name:extension Path:/usr/local/lib/docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.13] map[Name:sbom Path:/usr/local/lib/docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/usr/lo
cal/lib/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.21.0]] Warnings:<nil>}}
I0114 03:02:35.859162 16696 out.go:177] * Using the docker driver based on user configuration
I0114 03:02:35.895011 16696 start.go:294] selected driver: docker
I0114 03:02:35.895027 16696 start.go:838] validating driver "docker" against <nil>
I0114 03:02:35.895048 16696 start.go:849] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0114 03:02:35.897885 16696 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0114 03:02:36.071417 16696 info.go:266] docker info: {ID:VF6S:3GIL:4JQH:LPDQ:6EC6:D32C:6RZ7:IA3N:LZ7R:3YN2:QUOM:SIJ5 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:59 OomKillDisable:false NGoroutines:52 SystemTime:2023-01-14 11:02:35.956989466 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:3 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.21 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:1c90a442489720eec95342e1789ee8a5e1b9536f Expected:1c90a442489720eec95342e1789ee8a5e1b9536f} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/local/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1] map[Name:compose Path:/usr/local/lib/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.12.2] map[Name:dev Path:/usr/local/lib/docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.3] map[Name:extension Path:/usr/local/lib/docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.13] map[Name:sbom Path:/usr/local/lib/docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/usr/lo
cal/lib/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.21.0]] Warnings:<nil>}}
I0114 03:02:36.071608 16696 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0114 03:02:36.071782 16696 start_flags.go:917] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0114 03:02:36.095960 16696 out.go:177] * Using Docker Desktop driver with root privileges
I0114 03:02:36.117177 16696 cni.go:95] Creating CNI manager for ""
I0114 03:02:36.117198 16696 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0114 03:02:36.117219 16696 start_flags.go:319] config:
{Name:old-k8s-version-030235 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-030235 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:}
I0114 03:02:36.160110 16696 out.go:177] * Starting control plane node old-k8s-version-030235 in cluster old-k8s-version-030235
I0114 03:02:36.181105 16696 cache.go:120] Beginning downloading kic base image for docker with docker
I0114 03:02:36.202023 16696 out.go:177] * Pulling base image ...
I0114 03:02:36.244308 16696 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0114 03:02:36.244352 16696 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c in local docker daemon
I0114 03:02:36.244397 16696 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15642-1559/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0114 03:02:36.244414 16696 cache.go:57] Caching tarball of preloaded images
I0114 03:02:36.244664 16696 preload.go:174] Found /Users/jenkins/minikube-integration/15642-1559/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0114 03:02:36.244687 16696 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0114 03:02:36.245779 16696 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/config.json ...
I0114 03:02:36.245971 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/config.json: {Name:mk2fcf0f17202a0ae84341ab3578912698f667a1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:36.304254 16696 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c in local docker daemon, skipping pull
I0114 03:02:36.304282 16696 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c exists in daemon, skipping load
I0114 03:02:36.304310 16696 cache.go:193] Successfully downloaded all kic artifacts
I0114 03:02:36.304364 16696 start.go:364] acquiring machines lock for old-k8s-version-030235: {Name:mk0a4f570c8f2752e6db1ad5a8ffefc98930515a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0114 03:02:36.304533 16696 start.go:368] acquired machines lock for "old-k8s-version-030235" in 144.647µs
I0114 03:02:36.304579 16696 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-030235 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-030235 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0114 03:02:36.304667 16696 start.go:125] createHost starting for "" (driver="docker")
I0114 03:02:36.326607 16696 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0114 03:02:36.327009 16696 start.go:159] libmachine.API.Create for "old-k8s-version-030235" (driver="docker")
I0114 03:02:36.327073 16696 client.go:168] LocalClient.Create starting
I0114 03:02:36.327278 16696 main.go:134] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem
I0114 03:02:36.327367 16696 main.go:134] libmachine: Decoding PEM data...
I0114 03:02:36.327400 16696 main.go:134] libmachine: Parsing certificate...
I0114 03:02:36.327531 16696 main.go:134] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/cert.pem
I0114 03:02:36.327601 16696 main.go:134] libmachine: Decoding PEM data...
I0114 03:02:36.327619 16696 main.go:134] libmachine: Parsing certificate...
I0114 03:02:36.328486 16696 cli_runner.go:164] Run: docker network inspect old-k8s-version-030235 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0114 03:02:36.390198 16696 cli_runner.go:211] docker network inspect old-k8s-version-030235 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0114 03:02:36.390299 16696 network_create.go:280] running [docker network inspect old-k8s-version-030235] to gather additional debugging logs...
I0114 03:02:36.390317 16696 cli_runner.go:164] Run: docker network inspect old-k8s-version-030235
W0114 03:02:36.448533 16696 cli_runner.go:211] docker network inspect old-k8s-version-030235 returned with exit code 1
I0114 03:02:36.448557 16696 network_create.go:283] error running [docker network inspect old-k8s-version-030235]: docker network inspect old-k8s-version-030235: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-030235
I0114 03:02:36.448574 16696 network_create.go:285] output of [docker network inspect old-k8s-version-030235]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-030235
** /stderr **
I0114 03:02:36.448671 16696 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0114 03:02:36.509179 16696 network.go:277] reserving subnet 192.168.49.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.49.0:0xc0007e2710] misses:0}
I0114 03:02:36.509221 16696 network.go:210] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.509236 16696 network_create.go:123] attempt to create docker network old-k8s-version-030235 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 1500 ...
I0114 03:02:36.509338 16696 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235
W0114 03:02:36.569871 16696 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235 returned with exit code 1
W0114 03:02:36.569909 16696 network_create.go:115] failed to create docker network old-k8s-version-030235 192.168.49.0/24, will retry: subnet is taken
I0114 03:02:36.570195 16696 network.go:268] skipping subnet 192.168.49.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:false}} dirty:map[] misses:0}
I0114 03:02:36.570210 16696 network.go:213] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.570420 16696 network.go:277] reserving subnet 192.168.58.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:true}} dirty:map[192.168.49.0:0xc0007e2710 192.168.58.0:0xc0003c4500] misses:0}
I0114 03:02:36.570432 16696 network.go:210] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.570441 16696 network_create.go:123] attempt to create docker network old-k8s-version-030235 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 1500 ...
I0114 03:02:36.570529 16696 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235
W0114 03:02:36.626996 16696 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235 returned with exit code 1
W0114 03:02:36.627037 16696 network_create.go:115] failed to create docker network old-k8s-version-030235 192.168.58.0/24, will retry: subnet is taken
I0114 03:02:36.627306 16696 network.go:268] skipping subnet 192.168.58.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:true}} dirty:map[192.168.49.0:0xc0007e2710 192.168.58.0:0xc0003c4500] misses:1}
I0114 03:02:36.627324 16696 network.go:213] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.627542 16696 network.go:277] reserving subnet 192.168.67.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:true}} dirty:map[192.168.49.0:0xc0007e2710 192.168.58.0:0xc0003c4500 192.168.67.0:0xc0011540c8] misses:1}
I0114 03:02:36.627554 16696 network.go:210] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.627572 16696 network_create.go:123] attempt to create docker network old-k8s-version-030235 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 1500 ...
I0114 03:02:36.627675 16696 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235
W0114 03:02:36.688381 16696 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235 returned with exit code 1
W0114 03:02:36.688420 16696 network_create.go:115] failed to create docker network old-k8s-version-030235 192.168.67.0/24, will retry: subnet is taken
I0114 03:02:36.688675 16696 network.go:268] skipping subnet 192.168.67.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:true}} dirty:map[192.168.49.0:0xc0007e2710 192.168.58.0:0xc0003c4500 192.168.67.0:0xc0011540c8] misses:2}
I0114 03:02:36.688700 16696 network.go:213] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.688916 16696 network.go:277] reserving subnet 192.168.76.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc0007e2710] amended:true}} dirty:map[192.168.49.0:0xc0007e2710 192.168.58.0:0xc0003c4500 192.168.67.0:0xc0011540c8 192.168.76.0:0xc0003c4538] misses:2}
I0114 03:02:36.688930 16696 network.go:210] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0114 03:02:36.688939 16696 network_create.go:123] attempt to create docker network old-k8s-version-030235 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0114 03:02:36.689028 16696 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-030235 old-k8s-version-030235
I0114 03:02:36.785000 16696 network_create.go:107] docker network old-k8s-version-030235 192.168.76.0/24 created
I0114 03:02:36.785037 16696 kic.go:117] calculated static IP "192.168.76.2" for the "old-k8s-version-030235" container
I0114 03:02:36.785184 16696 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0114 03:02:36.844466 16696 cli_runner.go:164] Run: docker volume create old-k8s-version-030235 --label name.minikube.sigs.k8s.io=old-k8s-version-030235 --label created_by.minikube.sigs.k8s.io=true
I0114 03:02:36.905671 16696 oci.go:103] Successfully created a docker volume old-k8s-version-030235
I0114 03:02:36.905835 16696 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-030235-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-030235 --entrypoint /usr/bin/test -v old-k8s-version-030235:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c -d /var/lib
I0114 03:02:37.375264 16696 oci.go:107] Successfully prepared a docker volume old-k8s-version-030235
I0114 03:02:37.375311 16696 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0114 03:02:37.375326 16696 kic.go:190] Starting extracting preloaded images to volume ...
I0114 03:02:37.375437 16696 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15642-1559/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-030235:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c -I lz4 -xf /preloaded.tar -C /extractDir
I0114 03:02:44.260102 16696 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15642-1559/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-030235:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c -I lz4 -xf /preloaded.tar -C /extractDir: (6.884548087s)
I0114 03:02:44.260124 16696 kic.go:199] duration metric: took 6.884745 seconds to extract preloaded images to volume
I0114 03:02:44.260240 16696 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0114 03:02:44.411594 16696 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-030235 --name old-k8s-version-030235 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-030235 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-030235 --network old-k8s-version-030235 --ip 192.168.76.2 --volume old-k8s-version-030235:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=8443 --publish=22 --publish=2376 --publish=5000 --publish=32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c
I0114 03:02:44.861154 16696 cli_runner.go:164] Run: docker container inspect old-k8s-version-030235 --format={{.State.Running}}
I0114 03:02:44.928399 16696 cli_runner.go:164] Run: docker container inspect old-k8s-version-030235 --format={{.State.Status}}
I0114 03:02:45.000351 16696 cli_runner.go:164] Run: docker exec old-k8s-version-030235 stat /var/lib/dpkg/alternatives/iptables
I0114 03:02:45.119698 16696 oci.go:144] the created container "old-k8s-version-030235" has a running status.
I0114 03:02:45.119730 16696 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa...
I0114 03:02:45.195075 16696 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0114 03:02:45.308008 16696 cli_runner.go:164] Run: docker container inspect old-k8s-version-030235 --format={{.State.Status}}
I0114 03:02:45.371711 16696 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0114 03:02:45.371734 16696 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-030235 chown docker:docker /home/docker/.ssh/authorized_keys]
I0114 03:02:45.474910 16696 cli_runner.go:164] Run: docker container inspect old-k8s-version-030235 --format={{.State.Status}}
I0114 03:02:45.533100 16696 machine.go:88] provisioning docker machine ...
I0114 03:02:45.533145 16696 ubuntu.go:169] provisioning hostname "old-k8s-version-030235"
I0114 03:02:45.533262 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:45.599646 16696 main.go:134] libmachine: Using SSH client type: native
I0114 03:02:45.599850 16696 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec4a0] 0x13ef620 <nil> [] 0s} 127.0.0.1 53832 <nil> <nil>}
I0114 03:02:45.599869 16696 main.go:134] libmachine: About to run SSH command:
sudo hostname old-k8s-version-030235 && echo "old-k8s-version-030235" | sudo tee /etc/hostname
I0114 03:02:45.731016 16696 main.go:134] libmachine: SSH cmd err, output: <nil>: old-k8s-version-030235
I0114 03:02:45.731152 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:45.790834 16696 main.go:134] libmachine: Using SSH client type: native
I0114 03:02:45.791005 16696 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec4a0] 0x13ef620 <nil> [] 0s} 127.0.0.1 53832 <nil> <nil>}
I0114 03:02:45.791020 16696 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-030235' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-030235/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-030235' | sudo tee -a /etc/hosts;
fi
fi
I0114 03:02:45.908083 16696 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0114 03:02:45.908105 16696 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15642-1559/.minikube CaCertPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15642-1559/.minikube}
I0114 03:02:45.908126 16696 ubuntu.go:177] setting up certificates
I0114 03:02:45.908134 16696 provision.go:83] configureAuth start
I0114 03:02:45.908220 16696 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-030235
I0114 03:02:45.965245 16696 provision.go:138] copyHostCerts
I0114 03:02:45.965356 16696 exec_runner.go:144] found /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.pem, removing ...
I0114 03:02:45.965364 16696 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.pem
I0114 03:02:45.965487 16696 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.pem (1082 bytes)
I0114 03:02:45.965713 16696 exec_runner.go:144] found /Users/jenkins/minikube-integration/15642-1559/.minikube/cert.pem, removing ...
I0114 03:02:45.965719 16696 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15642-1559/.minikube/cert.pem
I0114 03:02:45.965782 16696 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15642-1559/.minikube/cert.pem (1123 bytes)
I0114 03:02:45.965940 16696 exec_runner.go:144] found /Users/jenkins/minikube-integration/15642-1559/.minikube/key.pem, removing ...
I0114 03:02:45.965946 16696 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15642-1559/.minikube/key.pem
I0114 03:02:45.966007 16696 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15642-1559/.minikube/key.pem (1679 bytes)
I0114 03:02:45.966140 16696 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-030235 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-030235]
I0114 03:02:46.112708 16696 provision.go:172] copyRemoteCerts
I0114 03:02:46.112779 16696 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0114 03:02:46.112843 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:46.172286 16696 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:53832 SSHKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa Username:docker}
I0114 03:02:46.259115 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0114 03:02:46.276445 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0114 03:02:46.293573 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0114 03:02:46.310987 16696 provision.go:86] duration metric: configureAuth took 402.837849ms
I0114 03:02:46.311004 16696 ubuntu.go:193] setting minikube options for container-runtime
I0114 03:02:46.311159 16696 config.go:180] Loaded profile config "old-k8s-version-030235": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0114 03:02:46.311236 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:46.370686 16696 main.go:134] libmachine: Using SSH client type: native
I0114 03:02:46.370849 16696 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec4a0] 0x13ef620 <nil> [] 0s} 127.0.0.1 53832 <nil> <nil>}
I0114 03:02:46.370863 16696 main.go:134] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0114 03:02:46.489526 16696 main.go:134] libmachine: SSH cmd err, output: <nil>: overlay
I0114 03:02:46.489547 16696 ubuntu.go:71] root file system type: overlay
I0114 03:02:46.489675 16696 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0114 03:02:46.489767 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:46.547705 16696 main.go:134] libmachine: Using SSH client type: native
I0114 03:02:46.547857 16696 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec4a0] 0x13ef620 <nil> [] 0s} 127.0.0.1 53832 <nil> <nil>}
I0114 03:02:46.547906 16696 main.go:134] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0114 03:02:46.674943 16696 main.go:134] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0114 03:02:46.675070 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:46.733223 16696 main.go:134] libmachine: Using SSH client type: native
I0114 03:02:46.733399 16696 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec4a0] 0x13ef620 <nil> [] 0s} 127.0.0.1 53832 <nil> <nil>}
I0114 03:02:46.733413 16696 main.go:134] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0114 03:02:47.318649 16696 main.go:134] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2022-10-25 18:00:04.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-01-14 11:02:46.672100181 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0114 03:02:47.318676 16696 machine.go:91] provisioned docker machine in 1.785544029s
I0114 03:02:47.318687 16696 client.go:171] LocalClient.Create took 10.99152392s
I0114 03:02:47.318705 16696 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-030235" took 10.991617979s
I0114 03:02:47.318715 16696 start.go:300] post-start starting for "old-k8s-version-030235" (driver="docker")
I0114 03:02:47.318721 16696 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0114 03:02:47.318796 16696 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0114 03:02:47.318860 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:47.378808 16696 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:53832 SSHKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa Username:docker}
I0114 03:02:47.466584 16696 ssh_runner.go:195] Run: cat /etc/os-release
I0114 03:02:47.470217 16696 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0114 03:02:47.470233 16696 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0114 03:02:47.470240 16696 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0114 03:02:47.470247 16696 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0114 03:02:47.470259 16696 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15642-1559/.minikube/addons for local assets ...
I0114 03:02:47.470343 16696 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15642-1559/.minikube/files for local assets ...
I0114 03:02:47.470509 16696 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15642-1559/.minikube/files/etc/ssl/certs/27282.pem -> 27282.pem in /etc/ssl/certs
I0114 03:02:47.470694 16696 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0114 03:02:47.478000 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/files/etc/ssl/certs/27282.pem --> /etc/ssl/certs/27282.pem (1708 bytes)
I0114 03:02:47.494992 16696 start.go:303] post-start completed in 176.259969ms
I0114 03:02:47.495560 16696 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-030235
I0114 03:02:47.553629 16696 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/config.json ...
I0114 03:02:47.554093 16696 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0114 03:02:47.554163 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:47.612526 16696 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:53832 SSHKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa Username:docker}
I0114 03:02:47.696942 16696 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0114 03:02:47.701781 16696 start.go:128] duration metric: createHost completed in 11.397017389s
I0114 03:02:47.701800 16696 start.go:83] releasing machines lock for "old-k8s-version-030235", held for 11.397169191s
I0114 03:02:47.701910 16696 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-030235
I0114 03:02:47.759059 16696 ssh_runner.go:195] Run: cat /version.json
I0114 03:02:47.759063 16696 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0114 03:02:47.759134 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:47.759161 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:47.822172 16696 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:53832 SSHKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa Username:docker}
I0114 03:02:47.823880 16696 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:53832 SSHKeyPath:/Users/jenkins/minikube-integration/15642-1559/.minikube/machines/old-k8s-version-030235/id_rsa Username:docker}
I0114 03:02:48.179241 16696 ssh_runner.go:195] Run: systemctl --version
I0114 03:02:48.183998 16696 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0114 03:02:48.193605 16696 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0114 03:02:48.193680 16696 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0114 03:02:48.203049 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0114 03:02:48.216386 16696 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0114 03:02:48.282870 16696 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0114 03:02:48.350776 16696 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0114 03:02:48.423813 16696 ssh_runner.go:195] Run: sudo systemctl restart docker
I0114 03:02:48.622658 16696 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0114 03:02:48.653145 16696 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0114 03:02:48.727806 16696 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 20.10.21 ...
I0114 03:02:48.727953 16696 cli_runner.go:164] Run: docker exec -t old-k8s-version-030235 dig +short host.docker.internal
I0114 03:02:48.839359 16696 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0114 03:02:48.839483 16696 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0114 03:02:48.843880 16696 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0114 03:02:48.853792 16696 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-030235
I0114 03:02:48.911804 16696 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0114 03:02:48.911885 16696 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0114 03:02:48.936391 16696 docker.go:613] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0114 03:02:48.936411 16696 docker.go:543] Images already preloaded, skipping extraction
I0114 03:02:48.936622 16696 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0114 03:02:48.960046 16696 docker.go:613] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0114 03:02:48.960065 16696 cache_images.go:84] Images are preloaded, skipping loading
I0114 03:02:48.960170 16696 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0114 03:02:49.038329 16696 cni.go:95] Creating CNI manager for ""
I0114 03:02:49.038349 16696 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0114 03:02:49.038366 16696 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0114 03:02:49.038381 16696 kubeadm.go:158] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-030235 NodeName:old-k8s-version-030235 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:systemd ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[]}
I0114 03:02:49.038507 16696 kubeadm.go:163] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-030235"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-030235
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: systemd
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0114 03:02:49.038589 16696 kubeadm.go:962] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-030235 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-030235 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0114 03:02:49.038658 16696 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0114 03:02:49.046720 16696 binaries.go:44] Found k8s binaries, skipping transfer
I0114 03:02:49.046785 16696 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0114 03:02:49.054485 16696 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0114 03:02:49.068875 16696 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0114 03:02:49.083596 16696 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2120 bytes)
I0114 03:02:49.096655 16696 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0114 03:02:49.100562 16696 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0114 03:02:49.110481 16696 certs.go:54] Setting up /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235 for IP: 192.168.76.2
I0114 03:02:49.110607 16696 certs.go:182] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.key
I0114 03:02:49.110661 16696 certs.go:182] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15642-1559/.minikube/proxy-client-ca.key
I0114 03:02:49.110713 16696 certs.go:302] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.key
I0114 03:02:49.110732 16696 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.crt with IP's: []
I0114 03:02:49.313517 16696 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.crt ...
I0114 03:02:49.313545 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.crt: {Name:mkf794a026450812db46a94fc137599df33bee27 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.313895 16696 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.key ...
I0114 03:02:49.313903 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/client.key: {Name:mk76ca06fbe3cc4f494315d4688c4ce01b6299ca Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.314118 16696 certs.go:302] generating minikube signed cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key.31bdca25
I0114 03:02:49.314137 16696 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0114 03:02:49.453171 16696 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt.31bdca25 ...
I0114 03:02:49.453191 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt.31bdca25: {Name:mk695e60715bda8f4ffa874f8bb8397b2de81bdb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.453501 16696 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key.31bdca25 ...
I0114 03:02:49.453509 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key.31bdca25: {Name:mk9c9f965c41d06f2db3d051be5d9827376b5d56 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.453918 16696 certs.go:320] copying /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt
I0114 03:02:49.454297 16696 certs.go:324] copying /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key
I0114 03:02:49.454512 16696 certs.go:302] generating aggregator signed cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.key
I0114 03:02:49.454530 16696 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.crt with IP's: []
I0114 03:02:49.545281 16696 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.crt ...
I0114 03:02:49.545301 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.crt: {Name:mk06c3de19a4f09ce2f2ef54556c398368712d4e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.545687 16696 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.key ...
I0114 03:02:49.545697 16696 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.key: {Name:mkeab6e4c7ee1921b2de275ae7671967aced2980 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0114 03:02:49.546296 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/2728.pem (1338 bytes)
W0114 03:02:49.546346 16696 certs.go:384] ignoring /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/2728_empty.pem, impossibly tiny 0 bytes
I0114 03:02:49.546372 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca-key.pem (1675 bytes)
I0114 03:02:49.546425 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/ca.pem (1082 bytes)
I0114 03:02:49.546468 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/cert.pem (1123 bytes)
I0114 03:02:49.546506 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/certs/key.pem (1679 bytes)
I0114 03:02:49.546591 16696 certs.go:388] found cert: /Users/jenkins/minikube-integration/15642-1559/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15642-1559/.minikube/files/etc/ssl/certs/27282.pem (1708 bytes)
I0114 03:02:49.547113 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0114 03:02:49.568587 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0114 03:02:49.588877 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0114 03:02:49.609799 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/profiles/old-k8s-version-030235/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0114 03:02:49.629539 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0114 03:02:49.649724 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0114 03:02:49.670458 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0114 03:02:49.690006 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0114 03:02:49.712107 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/files/etc/ssl/certs/27282.pem --> /usr/share/ca-certificates/27282.pem (1708 bytes)
I0114 03:02:49.734001 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0114 03:02:49.756889 16696 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15642-1559/.minikube/certs/2728.pem --> /usr/share/ca-certificates/2728.pem (1338 bytes)
I0114 03:02:49.778791 16696 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0114 03:02:49.793955 16696 ssh_runner.go:195] Run: openssl version
I0114 03:02:49.799557 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/27282.pem && ln -fs /usr/share/ca-certificates/27282.pem /etc/ssl/certs/27282.pem"
I0114 03:02:49.807772 16696 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/27282.pem
I0114 03:02:49.812053 16696 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Jan 14 10:11 /usr/share/ca-certificates/27282.pem
I0114 03:02:49.812104 16696 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/27282.pem
I0114 03:02:49.817527 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/27282.pem /etc/ssl/certs/3ec20f2e.0"
I0114 03:02:49.825953 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0114 03:02:49.836281 16696 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0114 03:02:49.842662 16696 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Jan 14 10:06 /usr/share/ca-certificates/minikubeCA.pem
I0114 03:02:49.842754 16696 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0114 03:02:49.850632 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0114 03:02:49.862490 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2728.pem && ln -fs /usr/share/ca-certificates/2728.pem /etc/ssl/certs/2728.pem"
I0114 03:02:49.872412 16696 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2728.pem
I0114 03:02:49.877509 16696 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Jan 14 10:11 /usr/share/ca-certificates/2728.pem
I0114 03:02:49.877575 16696 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2728.pem
I0114 03:02:49.884235 16696 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2728.pem /etc/ssl/certs/51391683.0"
I0114 03:02:49.895697 16696 kubeadm.go:396] StartCluster: {Name:old-k8s-version-030235 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-030235 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP:}
I0114 03:02:49.895867 16696 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0114 03:02:49.920140 16696 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0114 03:02:49.928868 16696 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0114 03:02:49.939837 16696 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0114 03:02:49.939927 16696 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0114 03:02:49.950975 16696 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0114 03:02:49.951018 16696 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0114 03:02:50.180266 16696 kubeadm.go:317] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0114 03:02:50.273754 16696 kubeadm.go:317] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
I0114 03:02:50.373500 16696 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0114 03:04:47.765858 16696 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0114 03:04:47.766009 16696 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0114 03:04:47.770700 16696 kubeadm.go:317] [init] Using Kubernetes version: v1.16.0
I0114 03:04:47.770777 16696 kubeadm.go:317] [preflight] Running pre-flight checks
I0114 03:04:47.770876 16696 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0114 03:04:47.771032 16696 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0114 03:04:47.771157 16696 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0114 03:04:47.771295 16696 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0114 03:04:47.771416 16696 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0114 03:04:47.771489 16696 kubeadm.go:317] [kubelet-start] Activating the kubelet service
I0114 03:04:47.771579 16696 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0114 03:04:47.796984 16696 out.go:204] - Generating certificates and keys ...
I0114 03:04:47.797047 16696 kubeadm.go:317] [certs] Using existing ca certificate authority
I0114 03:04:47.797101 16696 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0114 03:04:47.797159 16696 kubeadm.go:317] [certs] Generating "apiserver-kubelet-client" certificate and key
I0114 03:04:47.797225 16696 kubeadm.go:317] [certs] Generating "front-proxy-ca" certificate and key
I0114 03:04:47.797299 16696 kubeadm.go:317] [certs] Generating "front-proxy-client" certificate and key
I0114 03:04:47.797342 16696 kubeadm.go:317] [certs] Generating "etcd/ca" certificate and key
I0114 03:04:47.797390 16696 kubeadm.go:317] [certs] Generating "etcd/server" certificate and key
I0114 03:04:47.797499 16696 kubeadm.go:317] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0114 03:04:47.797554 16696 kubeadm.go:317] [certs] Generating "etcd/peer" certificate and key
I0114 03:04:47.797672 16696 kubeadm.go:317] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0114 03:04:47.797738 16696 kubeadm.go:317] [certs] Generating "etcd/healthcheck-client" certificate and key
I0114 03:04:47.797790 16696 kubeadm.go:317] [certs] Generating "apiserver-etcd-client" certificate and key
I0114 03:04:47.797841 16696 kubeadm.go:317] [certs] Generating "sa" key and public key
I0114 03:04:47.797892 16696 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0114 03:04:47.797927 16696 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0114 03:04:47.797967 16696 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0114 03:04:47.798010 16696 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0114 03:04:47.798060 16696 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0114 03:04:47.798132 16696 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0114 03:04:47.837980 16696 out.go:204] - Booting up control plane ...
I0114 03:04:47.838134 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0114 03:04:47.838251 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0114 03:04:47.838309 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0114 03:04:47.838370 16696 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0114 03:04:47.838642 16696 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0114 03:04:47.838721 16696 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0114 03:04:47.838835 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:04:47.839051 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:04:47.839145 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:04:47.839348 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:04:47.839427 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:04:47.839574 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:04:47.839635 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:04:47.839812 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:04:47.839940 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:04:47.840199 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:04:47.840210 16696 kubeadm.go:317]
I0114 03:04:47.840255 16696 kubeadm.go:317] Unfortunately, an error has occurred:
I0114 03:04:47.840305 16696 kubeadm.go:317] timed out waiting for the condition
I0114 03:04:47.840314 16696 kubeadm.go:317]
I0114 03:04:47.840364 16696 kubeadm.go:317] This error is likely caused by:
I0114 03:04:47.840452 16696 kubeadm.go:317] - The kubelet is not running
I0114 03:04:47.840653 16696 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0114 03:04:47.840680 16696 kubeadm.go:317]
I0114 03:04:47.840879 16696 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0114 03:04:47.840943 16696 kubeadm.go:317] - 'systemctl status kubelet'
I0114 03:04:47.840973 16696 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0114 03:04:47.840985 16696 kubeadm.go:317]
I0114 03:04:47.841102 16696 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0114 03:04:47.841242 16696 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0114 03:04:47.841328 16696 kubeadm.go:317] Here is one example how you may list all Kubernetes containers running in docker:
I0114 03:04:47.841375 16696 kubeadm.go:317] - 'docker ps -a | grep kube | grep -v pause'
I0114 03:04:47.841513 16696 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0114 03:04:47.841584 16696 kubeadm.go:317] - 'docker logs CONTAINERID'
W0114 03:04:47.841802 16696 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-030235 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0114 03:04:47.841866 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0114 03:04:48.299386 16696 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0114 03:04:48.314103 16696 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0114 03:04:48.314184 16696 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0114 03:04:48.325014 16696 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0114 03:04:48.325053 16696 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0114 03:04:48.405264 16696 kubeadm.go:317] [init] Using Kubernetes version: v1.16.0
I0114 03:04:48.405319 16696 kubeadm.go:317] [preflight] Running pre-flight checks
I0114 03:04:48.735759 16696 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0114 03:04:48.735953 16696 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0114 03:04:48.736118 16696 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0114 03:04:48.982356 16696 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0114 03:04:48.982999 16696 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0114 03:04:48.990355 16696 kubeadm.go:317] [kubelet-start] Activating the kubelet service
I0114 03:04:49.054660 16696 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0114 03:04:49.075176 16696 out.go:204] - Generating certificates and keys ...
I0114 03:04:49.075320 16696 kubeadm.go:317] [certs] Using existing ca certificate authority
I0114 03:04:49.075422 16696 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0114 03:04:49.075524 16696 kubeadm.go:317] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0114 03:04:49.075607 16696 kubeadm.go:317] [certs] Using existing front-proxy-ca certificate authority
I0114 03:04:49.075674 16696 kubeadm.go:317] [certs] Using existing front-proxy-client certificate and key on disk
I0114 03:04:49.075729 16696 kubeadm.go:317] [certs] Using existing etcd/ca certificate authority
I0114 03:04:49.075796 16696 kubeadm.go:317] [certs] Using existing etcd/server certificate and key on disk
I0114 03:04:49.075860 16696 kubeadm.go:317] [certs] Using existing etcd/peer certificate and key on disk
I0114 03:04:49.075933 16696 kubeadm.go:317] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0114 03:04:49.076007 16696 kubeadm.go:317] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0114 03:04:49.076035 16696 kubeadm.go:317] [certs] Using the existing "sa" key
I0114 03:04:49.076119 16696 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0114 03:04:49.248322 16696 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0114 03:04:49.365288 16696 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0114 03:04:49.561565 16696 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0114 03:04:49.737690 16696 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0114 03:04:49.738417 16696 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0114 03:04:49.760122 16696 out.go:204] - Booting up control plane ...
I0114 03:04:49.760228 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0114 03:04:49.760371 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0114 03:04:49.760486 16696 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0114 03:04:49.760597 16696 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0114 03:04:49.760733 16696 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0114 03:05:29.749856 16696 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0114 03:05:29.751049 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:05:29.751218 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:05:34.752010 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:05:34.752163 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:05:44.754775 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:05:44.754988 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:06:04.756119 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:06:04.756318 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:06:44.758196 16696 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0114 03:06:44.758423 16696 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0114 03:06:44.758432 16696 kubeadm.go:317]
I0114 03:06:44.758479 16696 kubeadm.go:317] Unfortunately, an error has occurred:
I0114 03:06:44.758531 16696 kubeadm.go:317] timed out waiting for the condition
I0114 03:06:44.758548 16696 kubeadm.go:317]
I0114 03:06:44.758583 16696 kubeadm.go:317] This error is likely caused by:
I0114 03:06:44.758618 16696 kubeadm.go:317] - The kubelet is not running
I0114 03:06:44.758767 16696 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0114 03:06:44.758788 16696 kubeadm.go:317]
I0114 03:06:44.758925 16696 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0114 03:06:44.758966 16696 kubeadm.go:317] - 'systemctl status kubelet'
I0114 03:06:44.758999 16696 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0114 03:06:44.759005 16696 kubeadm.go:317]
I0114 03:06:44.759093 16696 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0114 03:06:44.759176 16696 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0114 03:06:44.759273 16696 kubeadm.go:317] Here is one example how you may list all Kubernetes containers running in docker:
I0114 03:06:44.759332 16696 kubeadm.go:317] - 'docker ps -a | grep kube | grep -v pause'
I0114 03:06:44.759425 16696 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0114 03:06:44.759472 16696 kubeadm.go:317] - 'docker logs CONTAINERID'
I0114 03:06:44.762165 16696 kubeadm.go:317] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0114 03:06:44.762273 16696 kubeadm.go:317] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
I0114 03:06:44.762361 16696 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0114 03:06:44.762428 16696 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0114 03:06:44.762492 16696 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0114 03:06:44.762516 16696 kubeadm.go:398] StartCluster complete in 3m54.876027494s
I0114 03:06:44.762614 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0114 03:06:44.785263 16696 logs.go:274] 0 containers: []
W0114 03:06:44.785275 16696 logs.go:276] No container was found matching "kube-apiserver"
I0114 03:06:44.785360 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0114 03:06:44.809230 16696 logs.go:274] 0 containers: []
W0114 03:06:44.809243 16696 logs.go:276] No container was found matching "etcd"
I0114 03:06:44.809332 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0114 03:06:44.833287 16696 logs.go:274] 0 containers: []
W0114 03:06:44.833300 16696 logs.go:276] No container was found matching "coredns"
I0114 03:06:44.833382 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0114 03:06:44.856976 16696 logs.go:274] 0 containers: []
W0114 03:06:44.856989 16696 logs.go:276] No container was found matching "kube-scheduler"
I0114 03:06:44.857077 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0114 03:06:44.879880 16696 logs.go:274] 0 containers: []
W0114 03:06:44.879894 16696 logs.go:276] No container was found matching "kube-proxy"
I0114 03:06:44.879980 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0114 03:06:44.902794 16696 logs.go:274] 0 containers: []
W0114 03:06:44.902806 16696 logs.go:276] No container was found matching "kubernetes-dashboard"
I0114 03:06:44.902887 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0114 03:06:44.926990 16696 logs.go:274] 0 containers: []
W0114 03:06:44.927002 16696 logs.go:276] No container was found matching "storage-provisioner"
I0114 03:06:44.927091 16696 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0114 03:06:44.950334 16696 logs.go:274] 0 containers: []
W0114 03:06:44.950351 16696 logs.go:276] No container was found matching "kube-controller-manager"
I0114 03:06:44.950362 16696 logs.go:123] Gathering logs for container status ...
I0114 03:06:44.950371 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0114 03:06:47.000544 16696 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.050146381s)
I0114 03:06:47.000690 16696 logs.go:123] Gathering logs for kubelet ...
I0114 03:06:47.000697 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0114 03:06:47.039637 16696 logs.go:123] Gathering logs for dmesg ...
I0114 03:06:47.039651 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0114 03:06:47.051768 16696 logs.go:123] Gathering logs for describe nodes ...
I0114 03:06:47.051782 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0114 03:06:47.106488 16696 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0114 03:06:47.106500 16696 logs.go:123] Gathering logs for Docker ...
I0114 03:06:47.106506 16696 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
W0114 03:06:47.122012 16696 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0114 03:06:47.122033 16696 out.go:239] *
*
W0114 03:06:47.122153 16696 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0114 03:06:47.122166 16696 out.go:239] *
*
W0114 03:06:47.122813 16696 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0114 03:06:47.185359 16696 out.go:177]
W0114 03:06:47.228428 16696 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.21. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0114 03:06:47.228500 16696 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0114 03:06:47.228543 16696 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0114 03:06:47.249275 16696 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-030235 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-030235
helpers_test.go:235: (dbg) docker inspect old-k8s-version-030235:
-- stdout --
[
{
"Id": "d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942",
"Created": "2023-01-14T11:02:44.471910321Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 256371,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-01-14T11:02:44.850584543Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:fc27a248bc74ebfeee6e23949796c0207c7892e924b780fcd7204ed3e09ea2a3",
"ResolvConfPath": "/var/lib/docker/containers/d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942/hostname",
"HostsPath": "/var/lib/docker/containers/d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942/hosts",
"LogPath": "/var/lib/docker/containers/d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942/d977528adcbe29b4d7ab3f9e8738960d61890b45ab28eb5441685433d0ecd942-json.log",
"Name": "/old-k8s-version-030235",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-030235:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-030235",
"PortBindings": {
"22/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/ec3eccb64083cca531e170ed100ae49bcda69dc3a0b152caee88100d92223408-init/diff:/var/lib/docker/overlay2/74c9e0d36b5b0c73e7df7f4bce3bd0c3d02cf9dc383bffd6fbcff44769e0e62a/diff:/var/lib/docker/overlay2/ba601a6c163e2d067928a6364b090a9785c3dd2470d90823ce10e62a47aa569f/diff:/var/lib/docker/overlay2/80b54fffffd853e7ba8f14b1c1ac90a8b75fb31aafab2d53fe628cb592a95844/diff:/var/lib/docker/overlay2/02213d03e53450db4a2d492831eba720749d97435157430d240b760477b64c78/diff:/var/lib/docker/overlay2/e3727b5662aa5fdeeef9053112ad90fb2f9aaecbfeeddefa3efb066881ae1677/diff:/var/lib/docker/overlay2/685adc0695be0cb9862d43898ceae6e6a36c3cc98f04bc25e314797bed3b1d95/diff:/var/lib/docker/overlay2/7e133e132419c5ad6565f89b3ecfdf2c9fa038e5b9c39fe81c1269cfb6bb0d22/diff:/var/lib/docker/overlay2/c4d27ebf7e050a3aee0acccdadb92fc9390befadef2b0b13b9ebe87a2af3ef50/diff:/var/lib/docker/overlay2/0f07a86eba9c199451031724816d33cb5d2e19c401514edd8c1e392fd795f1e1/diff:/var/lib/docker/overlay2/a51cfe
8ee6145a30d356888e940bfdda67bc55c29f3972b35ae93dd989943b1c/diff:/var/lib/docker/overlay2/b155ac1a426201afe2af9fba8a7ebbecd3d8271f8613d0f53dac7bb190bc977f/diff:/var/lib/docker/overlay2/7c5cec64dde89a12b95bb1a0bca411b06b69201cfdb3cc4b46cb87a5bcff9a7f/diff:/var/lib/docker/overlay2/dd54bb055fc70a41daa3f3e950f4bdadd925db2c588d7d831edb4cbb176d30c7/diff:/var/lib/docker/overlay2/f58b39c756189e32d5b9c66b5c3861eabf5ab01ebc6179fec7210d414762bf45/diff:/var/lib/docker/overlay2/6458e00e4b79399a4860e78a572cd21fd47cbca2a54d189f34bd4a438145a6f5/diff:/var/lib/docker/overlay2/66427e9f49ff5383f9f819513857efb87ee3f880df33a86ac46ebc140ff172ed/diff:/var/lib/docker/overlay2/33f03d40d23c6a829c43633ba96c4058fbf09a4cf912eb51e0ca23a65574b0a7/diff:/var/lib/docker/overlay2/e68584e2b5a5a18fbd6edeeba6d80fe43e2199775b520878ca842d463078a2d1/diff:/var/lib/docker/overlay2/a2bfe134a89cb821f2c8e5ec6b42888d30fac6a9ed1aa4853476bb33cfe2e157/diff:/var/lib/docker/overlay2/f55951d7e041b300f9842916d51648285b79860a132d032d3c23b80af7c280fa/diff:/var/lib/d
ocker/overlay2/76cb0b8d6987165c472c0c9d54491045539294d203577a4ed7fac7f7cbbf0322/diff:/var/lib/docker/overlay2/a8f6d057d4938258302dd54e9a2e99732b4a2ac5c869366e93983e3e8890d432/diff:/var/lib/docker/overlay2/16bf4a461f9fe0edba90225f752527e534469b1bfbeb5bca6315512786340bfe/diff:/var/lib/docker/overlay2/2d022a51ddd598853537ff8fbeca5b94beff9d5d7e6ca81ffe011aa35121268a/diff:/var/lib/docker/overlay2/e30d56ebfba93be441f305b1938dd2d0f847f649922524ebef1fbe3e4b3b4bf9/diff:/var/lib/docker/overlay2/12df07bd2576a7b97f383aa3fcb2535f75a901953859063d9b65944d2dd0b152/diff:/var/lib/docker/overlay2/79e70748fe1267851a900b8bca2ab4e0b34e8163714fc440602d9e0273c93421/diff:/var/lib/docker/overlay2/c4fa6441d4ff7ce1be2072a8f61c5c495ff1785d9fee891191262b893a6eff63/diff:/var/lib/docker/overlay2/748980353d2fab0e6498a85b0c558d9eb7f34703302b21298c310b98dcf4d6f9/diff:/var/lib/docker/overlay2/48f823bc2f4741841d95ac4706f52fe9d01883bce998d5c999bdc363c838b1ee/diff:/var/lib/docker/overlay2/5f4f42c0e92359fc7ea2cf540120bd09407fd1d8dee5b56896919b39d3e
70033/diff:/var/lib/docker/overlay2/4a4066d1d0f42bb48af787d9f9bd115bacffde91f4ca8c20648dad3b25f904b6/diff:/var/lib/docker/overlay2/5f1054f553934c922e4dffc5c3804a5825ed249f7df9c3da31e2081145c8749a/diff:/var/lib/docker/overlay2/a6fe8ece465ba51837f6a88e28c3b571b632f0b223900278ac4a5f5dc0577520/diff:/var/lib/docker/overlay2/ee3e9af6d65fe9d2da423711b90ee171fd35422619c22b802d5fead4f861d921/diff:/var/lib/docker/overlay2/b353b985af8b2f665218f5af5e89cb642745824e2c3b51bfe3aa58c801823c46/diff:/var/lib/docker/overlay2/4411168ee372991c59d386d2ec200449c718a5343f5efa545ad9552a5c349310/diff:/var/lib/docker/overlay2/eeb668637d75a5802fe62d8a71458c68195302676ff09eb1e973d633e24e8588/diff:/var/lib/docker/overlay2/67b1dd580c0c0e994c4fe1233fef817d2c085438c80485c1f2eec64392c7b709/diff:/var/lib/docker/overlay2/1ae992d82b2e0a4c2a667c7d0d9e243efda7ee206e17c862bf093fa976667cc3/diff:/var/lib/docker/overlay2/ab6d393733a7abd2a9bd5612a0cef5adc3cded30c596c212828a8475c9c29779/diff:/var/lib/docker/overlay2/c927272ea82dc6bb318adcf8eb94099eece7af
9df7f454ff921048ba7ce589d2/diff:/var/lib/docker/overlay2/722309d1402eda210190af6c69b6f9998aff66e78e5bbc972ae865d10f0474d7/diff:/var/lib/docker/overlay2/c8a4e498ea2b5c051ced01db75d10e4ed1619bd3acc28c000789b600f8a7e23b/diff",
"MergedDir": "/var/lib/docker/overlay2/ec3eccb64083cca531e170ed100ae49bcda69dc3a0b152caee88100d92223408/merged",
"UpperDir": "/var/lib/docker/overlay2/ec3eccb64083cca531e170ed100ae49bcda69dc3a0b152caee88100d92223408/diff",
"WorkDir": "/var/lib/docker/overlay2/ec3eccb64083cca531e170ed100ae49bcda69dc3a0b152caee88100d92223408/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-030235",
"Source": "/var/lib/docker/volumes/old-k8s-version-030235/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-030235",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.36-1668787669-15272@sha256:06094fc04b5dc02fbf1e2de7723c2a6db5d24c21fd2ddda91f6daaf29038cd9c",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-030235",
"name.minikube.sigs.k8s.io": "old-k8s-version-030235",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "b393e8cc2ca9791c036b47e94c66bf02358a9f3b2722a25ce7542ec7cb04d83c",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "53832"
}
],
"2376/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "53833"
}
],
"32443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "53834"
}
],
"5000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "53835"
}
],
"8443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "53836"
}
]
},
"SandboxKey": "/var/run/docker/netns/b393e8cc2ca9",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-030235": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"d977528adcbe",
"old-k8s-version-030235"
],
"NetworkID": "ab958c8662819925836c350f1443c8060424291379d9dc2b6c89656fa5f7da2a",
"EndpointID": "a44f0767487daa905fe61590638004cbf71d83c3b900131448d936455bc13f58",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-030235 -n old-k8s-version-030235
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-030235 -n old-k8s-version-030235: exit status 6 (398.039032ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0114 03:06:47.793021 17603 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-030235" does not appear in /Users/jenkins/minikube-integration/15642-1559/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-030235" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (252.71s)