=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-136000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
=== CONT TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-136000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m11.262122154s)
-- stdout --
* [old-k8s-version-136000] minikube v1.29.0 on Darwin 13.2
- MINIKUBE_LOCATION=15770
- KUBECONFIG=/Users/jenkins/minikube-integration/15770-1719/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/15770-1719/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-136000 in cluster old-k8s-version-136000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 20.10.23 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0203 15:01:03.904411 19862 out.go:296] Setting OutFile to fd 1 ...
I0203 15:01:03.904578 19862 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0203 15:01:03.904583 19862 out.go:309] Setting ErrFile to fd 2...
I0203 15:01:03.904587 19862 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0203 15:01:03.904699 19862 root.go:334] Updating PATH: /Users/jenkins/minikube-integration/15770-1719/.minikube/bin
I0203 15:01:03.905236 19862 out.go:303] Setting JSON to false
I0203 15:01:03.923534 19862 start.go:125] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":3638,"bootTime":1675461625,"procs":380,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.2","kernelVersion":"22.3.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W0203 15:01:03.923622 19862 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0203 15:01:03.945758 19862 out.go:177] * [old-k8s-version-136000] minikube v1.29.0 on Darwin 13.2
I0203 15:01:03.988429 19862 notify.go:220] Checking for updates...
I0203 15:01:04.010159 19862 out.go:177] - MINIKUBE_LOCATION=15770
I0203 15:01:04.031430 19862 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/15770-1719/kubeconfig
I0203 15:01:04.052753 19862 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0203 15:01:04.074565 19862 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0203 15:01:04.096365 19862 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/15770-1719/.minikube
I0203 15:01:04.117561 19862 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0203 15:01:04.140215 19862 config.go:180] Loaded profile config "false-292000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.26.1
I0203 15:01:04.140332 19862 driver.go:365] Setting default libvirt URI to qemu:///system
I0203 15:01:04.203489 19862 docker.go:141] docker version: linux-20.10.22:Docker Desktop 4.16.2 (95914)
I0203 15:01:04.203615 19862 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0203 15:01:04.348511 19862 info.go:266] docker info: {ID:GSNP:GK6O:NBBA:CS3H:B4YR:6KQI:MMNQ:OHLJ:PBZ2:MCN2:S4BS:ZXUA Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:61 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-03 23:01:04.254011652 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0203 15:01:04.370492 19862 out.go:177] * Using the docker driver based on user configuration
I0203 15:01:04.392408 19862 start.go:296] selected driver: docker
I0203 15:01:04.392438 19862 start.go:857] validating driver "docker" against <nil>
I0203 15:01:04.392456 19862 start.go:868] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0203 15:01:04.396471 19862 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0203 15:01:04.540788 19862 info.go:266] docker info: {ID:GSNP:GK6O:NBBA:CS3H:B4YR:6KQI:MMNQ:OHLJ:PBZ2:MCN2:S4BS:ZXUA Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:61 OomKillDisable:false NGoroutines:56 SystemTime:2023-02-03 23:01:04.446030372 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:5 KernelVersion:5.15.49-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.22 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9ba4b250366a5ddde94bb7c9d1def331423aa323 Expected:9ba4b250366a5ddde94bb7c9d1def331423aa323} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defaul
t name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.10.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.15.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.0.5] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.17] map[Name:sbom Path
:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I0203 15:01:04.540915 19862 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0203 15:01:04.541101 19862 start_flags.go:917] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0203 15:01:04.563043 19862 out.go:177] * Using Docker Desktop driver with root privileges
I0203 15:01:04.584469 19862 cni.go:84] Creating CNI manager for ""
I0203 15:01:04.584507 19862 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0203 15:01:04.584520 19862 start_flags.go:319] config:
{Name:old-k8s-version-136000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-136000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0203 15:01:04.628541 19862 out.go:177] * Starting control plane node old-k8s-version-136000 in cluster old-k8s-version-136000
I0203 15:01:04.649495 19862 cache.go:120] Beginning downloading kic base image for docker with docker
I0203 15:01:04.670530 19862 out.go:177] * Pulling base image ...
I0203 15:01:04.712565 19862 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0203 15:01:04.712619 19862 image.go:77] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 in local docker daemon
I0203 15:01:04.712659 19862 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/15770-1719/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0203 15:01:04.712675 19862 cache.go:57] Caching tarball of preloaded images
I0203 15:01:04.712885 19862 preload.go:174] Found /Users/jenkins/minikube-integration/15770-1719/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0203 15:01:04.712906 19862 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0203 15:01:04.713917 19862 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/config.json ...
I0203 15:01:04.714071 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/config.json: {Name:mk790fa0e2925daea627be1ef43250ed624d5ab1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:04.769795 19862 image.go:81] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 in local docker daemon, skipping pull
I0203 15:01:04.769810 19862 cache.go:143] gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 exists in daemon, skipping load
I0203 15:01:04.769830 19862 cache.go:193] Successfully downloaded all kic artifacts
I0203 15:01:04.769926 19862 start.go:364] acquiring machines lock for old-k8s-version-136000: {Name:mk6d4a37aad431df09b59c262f13f34239bde2da Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0203 15:01:04.770074 19862 start.go:368] acquired machines lock for "old-k8s-version-136000" in 135.335µs
I0203 15:01:04.770099 19862 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-136000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-136000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0203 15:01:04.770157 19862 start.go:125] createHost starting for "" (driver="docker")
I0203 15:01:04.813911 19862 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0203 15:01:04.814408 19862 start.go:159] libmachine.API.Create for "old-k8s-version-136000" (driver="docker")
I0203 15:01:04.814473 19862 client.go:168] LocalClient.Create starting
I0203 15:01:04.814718 19862 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem
I0203 15:01:04.814804 19862 main.go:141] libmachine: Decoding PEM data...
I0203 15:01:04.814836 19862 main.go:141] libmachine: Parsing certificate...
I0203 15:01:04.814933 19862 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/cert.pem
I0203 15:01:04.814994 19862 main.go:141] libmachine: Decoding PEM data...
I0203 15:01:04.815009 19862 main.go:141] libmachine: Parsing certificate...
I0203 15:01:04.815868 19862 cli_runner.go:164] Run: docker network inspect old-k8s-version-136000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0203 15:01:04.871506 19862 cli_runner.go:211] docker network inspect old-k8s-version-136000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0203 15:01:04.871601 19862 network_create.go:281] running [docker network inspect old-k8s-version-136000] to gather additional debugging logs...
I0203 15:01:04.871628 19862 cli_runner.go:164] Run: docker network inspect old-k8s-version-136000
W0203 15:01:04.926856 19862 cli_runner.go:211] docker network inspect old-k8s-version-136000 returned with exit code 1
I0203 15:01:04.926886 19862 network_create.go:284] error running [docker network inspect old-k8s-version-136000]: docker network inspect old-k8s-version-136000: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-136000
I0203 15:01:04.926896 19862 network_create.go:286] output of [docker network inspect old-k8s-version-136000]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-136000
** /stderr **
I0203 15:01:04.926989 19862 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0203 15:01:04.984433 19862 network.go:209] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0203 15:01:04.985136 19862 network.go:206] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc001248660}
I0203 15:01:04.985332 19862 network_create.go:123] attempt to create docker network old-k8s-version-136000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 1500 ...
I0203 15:01:04.985424 19862 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-136000 old-k8s-version-136000
W0203 15:01:05.040342 19862 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-136000 old-k8s-version-136000 returned with exit code 1
W0203 15:01:05.040374 19862 network_create.go:148] failed to create docker network old-k8s-version-136000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 1500: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-136000 old-k8s-version-136000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0203 15:01:05.040395 19862 network_create.go:115] failed to create docker network old-k8s-version-136000 192.168.58.0/24, will retry: subnet is taken
I0203 15:01:05.041872 19862 network.go:209] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0203 15:01:05.042185 19862 network.go:206] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000fc7eb0}
I0203 15:01:05.042195 19862 network_create.go:123] attempt to create docker network old-k8s-version-136000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 1500 ...
I0203 15:01:05.042266 19862 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-136000 old-k8s-version-136000
I0203 15:01:05.129984 19862 network_create.go:107] docker network old-k8s-version-136000 192.168.67.0/24 created
I0203 15:01:05.130028 19862 kic.go:117] calculated static IP "192.168.67.2" for the "old-k8s-version-136000" container
I0203 15:01:05.130160 19862 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0203 15:01:05.186491 19862 cli_runner.go:164] Run: docker volume create old-k8s-version-136000 --label name.minikube.sigs.k8s.io=old-k8s-version-136000 --label created_by.minikube.sigs.k8s.io=true
I0203 15:01:05.241890 19862 oci.go:103] Successfully created a docker volume old-k8s-version-136000
I0203 15:01:05.242036 19862 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-136000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-136000 --entrypoint /usr/bin/test -v old-k8s-version-136000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 -d /var/lib
I0203 15:01:05.706911 19862 oci.go:107] Successfully prepared a docker volume old-k8s-version-136000
I0203 15:01:05.706938 19862 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0203 15:01:05.706953 19862 kic.go:190] Starting extracting preloaded images to volume ...
I0203 15:01:05.707080 19862 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15770-1719/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-136000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 -I lz4 -xf /preloaded.tar -C /extractDir
I0203 15:01:11.692968 19862 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/15770-1719/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-136000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 -I lz4 -xf /preloaded.tar -C /extractDir: (5.985693089s)
I0203 15:01:11.692988 19862 kic.go:199] duration metric: took 5.985911 seconds to extract preloaded images to volume
I0203 15:01:11.693092 19862 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0203 15:01:11.835319 19862 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-136000 --name old-k8s-version-136000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-136000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-136000 --network old-k8s-version-136000 --ip 192.168.67.2 --volume old-k8s-version-136000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8
I0203 15:01:12.195277 19862 cli_runner.go:164] Run: docker container inspect old-k8s-version-136000 --format={{.State.Running}}
I0203 15:01:12.255838 19862 cli_runner.go:164] Run: docker container inspect old-k8s-version-136000 --format={{.State.Status}}
I0203 15:01:12.321819 19862 cli_runner.go:164] Run: docker exec old-k8s-version-136000 stat /var/lib/dpkg/alternatives/iptables
I0203 15:01:12.443806 19862 oci.go:144] the created container "old-k8s-version-136000" has a running status.
I0203 15:01:12.443837 19862 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa...
I0203 15:01:12.606759 19862 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0203 15:01:12.734044 19862 cli_runner.go:164] Run: docker container inspect old-k8s-version-136000 --format={{.State.Status}}
I0203 15:01:12.798725 19862 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0203 15:01:12.798748 19862 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-136000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0203 15:01:12.922032 19862 cli_runner.go:164] Run: docker container inspect old-k8s-version-136000 --format={{.State.Status}}
I0203 15:01:12.987845 19862 machine.go:88] provisioning docker machine ...
I0203 15:01:12.987905 19862 ubuntu.go:169] provisioning hostname "old-k8s-version-136000"
I0203 15:01:12.988026 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:13.052801 19862 main.go:141] libmachine: Using SSH client type: native
I0203 15:01:13.053008 19862 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55132 <nil> <nil>}
I0203 15:01:13.053020 19862 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-136000 && echo "old-k8s-version-136000" | sudo tee /etc/hostname
I0203 15:01:13.195071 19862 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-136000
I0203 15:01:13.195168 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:13.257534 19862 main.go:141] libmachine: Using SSH client type: native
I0203 15:01:13.257715 19862 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55132 <nil> <nil>}
I0203 15:01:13.257733 19862 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-136000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-136000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-136000' | sudo tee -a /etc/hosts;
fi
fi
I0203 15:01:13.385205 19862 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0203 15:01:13.385229 19862 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/15770-1719/.minikube CaCertPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/15770-1719/.minikube}
I0203 15:01:13.385254 19862 ubuntu.go:177] setting up certificates
I0203 15:01:13.385272 19862 provision.go:83] configureAuth start
I0203 15:01:13.385354 19862 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-136000
I0203 15:01:13.445028 19862 provision.go:138] copyHostCerts
I0203 15:01:13.445121 19862 exec_runner.go:144] found /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.pem, removing ...
I0203 15:01:13.445128 19862 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.pem
I0203 15:01:13.445277 19862 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.pem (1078 bytes)
I0203 15:01:13.445511 19862 exec_runner.go:144] found /Users/jenkins/minikube-integration/15770-1719/.minikube/cert.pem, removing ...
I0203 15:01:13.445517 19862 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15770-1719/.minikube/cert.pem
I0203 15:01:13.445593 19862 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/15770-1719/.minikube/cert.pem (1123 bytes)
I0203 15:01:13.445752 19862 exec_runner.go:144] found /Users/jenkins/minikube-integration/15770-1719/.minikube/key.pem, removing ...
I0203 15:01:13.445758 19862 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/15770-1719/.minikube/key.pem
I0203 15:01:13.445824 19862 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/15770-1719/.minikube/key.pem (1675 bytes)
I0203 15:01:13.445937 19862 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-136000 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-136000]
I0203 15:01:13.648041 19862 provision.go:172] copyRemoteCerts
I0203 15:01:13.648099 19862 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0203 15:01:13.648150 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:13.708333 19862 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55132 SSHKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa Username:docker}
I0203 15:01:13.799073 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0203 15:01:13.816941 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0203 15:01:13.836231 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0203 15:01:13.854607 19862 provision.go:86] duration metric: configureAuth took 469.312672ms
I0203 15:01:13.854620 19862 ubuntu.go:193] setting minikube options for container-runtime
I0203 15:01:13.854767 19862 config.go:180] Loaded profile config "old-k8s-version-136000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0203 15:01:13.854835 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:13.943052 19862 main.go:141] libmachine: Using SSH client type: native
I0203 15:01:13.943192 19862 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55132 <nil> <nil>}
I0203 15:01:13.943208 19862 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0203 15:01:14.072694 19862 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0203 15:01:14.072708 19862 ubuntu.go:71] root file system type: overlay
I0203 15:01:14.072957 19862 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0203 15:01:14.073064 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:14.136496 19862 main.go:141] libmachine: Using SSH client type: native
I0203 15:01:14.136687 19862 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55132 <nil> <nil>}
I0203 15:01:14.136750 19862 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0203 15:01:14.271153 19862 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0203 15:01:14.271253 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:14.335290 19862 main.go:141] libmachine: Using SSH client type: native
I0203 15:01:14.335437 19862 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13ec500] 0x13ef680 <nil> [] 0s} 127.0.0.1 55132 <nil> <nil>}
I0203 15:01:14.335449 19862 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0203 15:01:14.970874 19862 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-01-19 17:34:14.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-02-03 23:01:14.268819802 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0203 15:01:14.970900 19862 machine.go:91] provisioned docker machine in 1.982974492s
I0203 15:01:14.970913 19862 client.go:171] LocalClient.Create took 10.156216583s
I0203 15:01:14.970935 19862 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-136000" took 10.156318936s
I0203 15:01:14.970942 19862 start.go:300] post-start starting for "old-k8s-version-136000" (driver="docker")
I0203 15:01:14.970946 19862 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0203 15:01:14.971021 19862 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0203 15:01:14.971096 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:15.036412 19862 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55132 SSHKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa Username:docker}
I0203 15:01:15.130005 19862 ssh_runner.go:195] Run: cat /etc/os-release
I0203 15:01:15.134057 19862 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0203 15:01:15.134075 19862 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0203 15:01:15.134082 19862 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0203 15:01:15.134086 19862 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0203 15:01:15.134097 19862 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15770-1719/.minikube/addons for local assets ...
I0203 15:01:15.134218 19862 filesync.go:126] Scanning /Users/jenkins/minikube-integration/15770-1719/.minikube/files for local assets ...
I0203 15:01:15.134394 19862 filesync.go:149] local asset: /Users/jenkins/minikube-integration/15770-1719/.minikube/files/etc/ssl/certs/25682.pem -> 25682.pem in /etc/ssl/certs
I0203 15:01:15.134596 19862 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0203 15:01:15.142398 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/files/etc/ssl/certs/25682.pem --> /etc/ssl/certs/25682.pem (1708 bytes)
I0203 15:01:15.159787 19862 start.go:303] post-start completed in 188.828643ms
I0203 15:01:15.160294 19862 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-136000
I0203 15:01:15.220370 19862 profile.go:148] Saving config to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/config.json ...
I0203 15:01:15.220911 19862 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0203 15:01:15.220999 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:15.281557 19862 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55132 SSHKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa Username:docker}
I0203 15:01:15.372411 19862 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0203 15:01:15.377695 19862 start.go:128] duration metric: createHost completed in 10.60730173s
I0203 15:01:15.377718 19862 start.go:83] releasing machines lock for "old-k8s-version-136000", held for 10.607414291s
I0203 15:01:15.377823 19862 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-136000
I0203 15:01:15.446571 19862 ssh_runner.go:195] Run: cat /version.json
I0203 15:01:15.446606 19862 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0203 15:01:15.446677 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:15.446686 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:15.520897 19862 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55132 SSHKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa Username:docker}
I0203 15:01:15.520906 19862 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:55132 SSHKeyPath:/Users/jenkins/minikube-integration/15770-1719/.minikube/machines/old-k8s-version-136000/id_rsa Username:docker}
I0203 15:01:15.805895 19862 ssh_runner.go:195] Run: systemctl --version
I0203 15:01:15.810937 19862 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0203 15:01:15.816451 19862 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0203 15:01:15.839770 19862 cni.go:229] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0203 15:01:15.839852 19862 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0203 15:01:15.856786 19862 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0203 15:01:15.866169 19862 cni.go:307] configured [/etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0203 15:01:15.866189 19862 start.go:483] detecting cgroup driver to use...
I0203 15:01:15.866203 19862 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0203 15:01:15.866350 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0203 15:01:15.882007 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "k8s.gcr.io/pause:3.1"|' /etc/containerd/config.toml"
I0203 15:01:15.891878 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0203 15:01:15.902509 19862 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0203 15:01:15.902582 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0203 15:01:15.913482 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0203 15:01:15.923202 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0203 15:01:15.932774 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0203 15:01:15.942169 19862 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0203 15:01:15.951062 19862 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0203 15:01:15.960597 19862 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0203 15:01:15.970706 19862 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0203 15:01:15.979276 19862 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0203 15:01:16.051611 19862 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0203 15:01:16.126446 19862 start.go:483] detecting cgroup driver to use...
I0203 15:01:16.126467 19862 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0203 15:01:16.126549 19862 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0203 15:01:16.142787 19862 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0203 15:01:16.142855 19862 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0203 15:01:16.156405 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0203 15:01:16.170796 19862 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0203 15:01:16.275182 19862 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0203 15:01:16.375162 19862 docker.go:529] configuring docker to use "cgroupfs" as cgroup driver...
I0203 15:01:16.375179 19862 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0203 15:01:16.391556 19862 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0203 15:01:16.489323 19862 ssh_runner.go:195] Run: sudo systemctl restart docker
I0203 15:01:16.733325 19862 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0203 15:01:16.768102 19862 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0203 15:01:16.842909 19862 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 20.10.23 ...
I0203 15:01:16.843137 19862 cli_runner.go:164] Run: docker exec -t old-k8s-version-136000 dig +short host.docker.internal
I0203 15:01:16.966977 19862 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0203 15:01:16.967115 19862 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0203 15:01:16.972308 19862 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0203 15:01:16.983843 19862 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-136000
I0203 15:01:17.043620 19862 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0203 15:01:17.043702 19862 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0203 15:01:17.069439 19862 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0203 15:01:17.069456 19862 docker.go:560] Images already preloaded, skipping extraction
I0203 15:01:17.069544 19862 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0203 15:01:17.095069 19862 docker.go:630] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0203 15:01:17.095093 19862 cache_images.go:84] Images are preloaded, skipping loading
I0203 15:01:17.095177 19862 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0203 15:01:17.170817 19862 cni.go:84] Creating CNI manager for ""
I0203 15:01:17.170834 19862 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0203 15:01:17.170854 19862 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0203 15:01:17.170870 19862 kubeadm.go:172] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-136000 NodeName:old-k8s-version-136000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m]}
I0203 15:01:17.170984 19862 kubeadm.go:177] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-136000"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-136000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0203 15:01:17.171052 19862 kubeadm.go:968] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-136000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-136000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0203 15:01:17.171101 19862 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0203 15:01:17.180980 19862 binaries.go:44] Found k8s binaries, skipping transfer
I0203 15:01:17.181052 19862 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0203 15:01:17.190150 19862 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0203 15:01:17.205131 19862 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0203 15:01:17.220170 19862 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0203 15:01:17.235298 19862 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0203 15:01:17.239682 19862 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0203 15:01:17.251307 19862 certs.go:56] Setting up /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000 for IP: 192.168.67.2
I0203 15:01:17.251326 19862 certs.go:186] acquiring lock for shared ca certs: {Name:mkdec04c6cc16ac0dcab0ae849b602e6c1942576 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.251531 19862 certs.go:195] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.key
I0203 15:01:17.251598 19862 certs.go:195] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/15770-1719/.minikube/proxy-client-ca.key
I0203 15:01:17.251645 19862 certs.go:315] generating minikube-user signed cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.key
I0203 15:01:17.251658 19862 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.crt with IP's: []
I0203 15:01:17.308371 19862 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.crt ...
I0203 15:01:17.308393 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.crt: {Name:mkcd67f923f0e04e9cb8601b4f51bdbba081a5b8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.308744 19862 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.key ...
I0203 15:01:17.308753 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/client.key: {Name:mk70ceb7b3519640f17dd771d0c2bd69ce7cc490 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.308997 19862 certs.go:315] generating minikube signed cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key.c7fa3a9e
I0203 15:01:17.309013 19862 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt.c7fa3a9e with IP's: [192.168.67.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0203 15:01:17.493340 19862 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt.c7fa3a9e ...
I0203 15:01:17.493354 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt.c7fa3a9e: {Name:mk4c68462e1ad59d00cde0439cfe934ca2ae3f94 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.493649 19862 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key.c7fa3a9e ...
I0203 15:01:17.493657 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key.c7fa3a9e: {Name:mk6043d01e63efdbb835239f8296706533d4c028 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.493845 19862 certs.go:333] copying /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt.c7fa3a9e -> /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt
I0203 15:01:17.494026 19862 certs.go:337] copying /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key.c7fa3a9e -> /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key
I0203 15:01:17.494176 19862 certs.go:315] generating aggregator signed cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.key
I0203 15:01:17.494190 19862 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.crt with IP's: []
I0203 15:01:17.607914 19862 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.crt ...
I0203 15:01:17.607928 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.crt: {Name:mkae54fd2748717ce4cbf8a7ee086e5a32dab62c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.608271 19862 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.key ...
I0203 15:01:17.608283 19862 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.key: {Name:mka343fe3e9552cc05d63d3c04c58596827e0204 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0203 15:01:17.608725 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/2568.pem (1338 bytes)
W0203 15:01:17.608775 19862 certs.go:397] ignoring /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/2568_empty.pem, impossibly tiny 0 bytes
I0203 15:01:17.608786 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca-key.pem (1675 bytes)
I0203 15:01:17.608820 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/ca.pem (1078 bytes)
I0203 15:01:17.608850 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/cert.pem (1123 bytes)
I0203 15:01:17.608881 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/certs/key.pem (1675 bytes)
I0203 15:01:17.608952 19862 certs.go:401] found cert: /Users/jenkins/minikube-integration/15770-1719/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/15770-1719/.minikube/files/etc/ssl/certs/25682.pem (1708 bytes)
I0203 15:01:17.609472 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0203 15:01:17.628864 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0203 15:01:17.648306 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0203 15:01:17.668005 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/old-k8s-version-136000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0203 15:01:17.687979 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0203 15:01:17.707296 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0203 15:01:17.727205 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0203 15:01:17.746651 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0203 15:01:17.767003 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0203 15:01:17.786956 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/certs/2568.pem --> /usr/share/ca-certificates/2568.pem (1338 bytes)
I0203 15:01:17.805797 19862 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/15770-1719/.minikube/files/etc/ssl/certs/25682.pem --> /usr/share/ca-certificates/25682.pem (1708 bytes)
I0203 15:01:17.825589 19862 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0203 15:01:17.840730 19862 ssh_runner.go:195] Run: openssl version
I0203 15:01:17.847780 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/25682.pem && ln -fs /usr/share/ca-certificates/25682.pem /etc/ssl/certs/25682.pem"
I0203 15:01:17.857683 19862 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/25682.pem
I0203 15:01:17.862131 19862 certs.go:444] hashing: -rw-r--r-- 1 root root 1708 Feb 3 22:13 /usr/share/ca-certificates/25682.pem
I0203 15:01:17.862185 19862 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/25682.pem
I0203 15:01:17.868421 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/25682.pem /etc/ssl/certs/3ec20f2e.0"
I0203 15:01:17.877780 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0203 15:01:17.887295 19862 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0203 15:01:17.892630 19862 certs.go:444] hashing: -rw-r--r-- 1 root root 1111 Feb 3 22:08 /usr/share/ca-certificates/minikubeCA.pem
I0203 15:01:17.892697 19862 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0203 15:01:17.899298 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0203 15:01:17.908352 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2568.pem && ln -fs /usr/share/ca-certificates/2568.pem /etc/ssl/certs/2568.pem"
I0203 15:01:17.917372 19862 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2568.pem
I0203 15:01:17.921625 19862 certs.go:444] hashing: -rw-r--r-- 1 root root 1338 Feb 3 22:13 /usr/share/ca-certificates/2568.pem
I0203 15:01:17.921677 19862 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2568.pem
I0203 15:01:17.927528 19862 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2568.pem /etc/ssl/certs/51391683.0"
I0203 15:01:17.936330 19862 kubeadm.go:401] StartCluster: {Name:old-k8s-version-136000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-136000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0203 15:01:17.936447 19862 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0203 15:01:17.960144 19862 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0203 15:01:17.968323 19862 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0203 15:01:17.975885 19862 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0203 15:01:17.975943 19862 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0203 15:01:17.984297 19862 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0203 15:01:17.984320 19862 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0203 15:01:18.035727 19862 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0203 15:01:18.035771 19862 kubeadm.go:322] [preflight] Running pre-flight checks
I0203 15:01:18.388563 19862 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0203 15:01:18.388681 19862 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0203 15:01:18.388773 19862 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0203 15:01:18.639046 19862 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0203 15:01:18.640145 19862 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0203 15:01:18.646775 19862 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0203 15:01:18.713795 19862 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0203 15:01:18.756068 19862 out.go:204] - Generating certificates and keys ...
I0203 15:01:18.756169 19862 kubeadm.go:322] [certs] Using existing ca certificate authority
I0203 15:01:18.756261 19862 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0203 15:01:18.780053 19862 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0203 15:01:18.933774 19862 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0203 15:01:19.265340 19862 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0203 15:01:19.363543 19862 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0203 15:01:19.613185 19862 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0203 15:01:19.613325 19862 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0203 15:01:19.705479 19862 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0203 15:01:19.705922 19862 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0203 15:01:19.767479 19862 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0203 15:01:20.038350 19862 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0203 15:01:20.195253 19862 kubeadm.go:322] [certs] Generating "sa" key and public key
I0203 15:01:20.195321 19862 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0203 15:01:20.310360 19862 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0203 15:01:20.420421 19862 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0203 15:01:20.532378 19862 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0203 15:01:20.760933 19862 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0203 15:01:20.761544 19862 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0203 15:01:20.783222 19862 out.go:204] - Booting up control plane ...
I0203 15:01:20.783299 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0203 15:01:20.783377 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0203 15:01:20.783462 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0203 15:01:20.783534 19862 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0203 15:01:20.783655 19862 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0203 15:02:00.772022 19862 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0203 15:02:00.773035 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:02:00.773265 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:02:05.774183 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:02:05.774363 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:02:15.775108 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:02:15.775248 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:02:35.777566 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:02:35.777845 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:03:15.787487 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:03:15.787744 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:03:15.787762 19862 kubeadm.go:322]
I0203 15:03:15.787853 19862 kubeadm.go:322] Unfortunately, an error has occurred:
I0203 15:03:15.787891 19862 kubeadm.go:322] timed out waiting for the condition
I0203 15:03:15.787897 19862 kubeadm.go:322]
I0203 15:03:15.787923 19862 kubeadm.go:322] This error is likely caused by:
I0203 15:03:15.787961 19862 kubeadm.go:322] - The kubelet is not running
I0203 15:03:15.788068 19862 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0203 15:03:15.788078 19862 kubeadm.go:322]
I0203 15:03:15.788170 19862 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0203 15:03:15.788201 19862 kubeadm.go:322] - 'systemctl status kubelet'
I0203 15:03:15.788222 19862 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0203 15:03:15.788225 19862 kubeadm.go:322]
I0203 15:03:15.788313 19862 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0203 15:03:15.788387 19862 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0203 15:03:15.788456 19862 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0203 15:03:15.788501 19862 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0203 15:03:15.788573 19862 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0203 15:03:15.788604 19862 kubeadm.go:322] - 'docker logs CONTAINERID'
I0203 15:03:15.791394 19862 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0203 15:03:15.791467 19862 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0203 15:03:15.791580 19862 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
I0203 15:03:15.791668 19862 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0203 15:03:15.791732 19862 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0203 15:03:15.791810 19862 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0203 15:03:15.791966 19862 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-136000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0203 15:03:15.791997 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0203 15:03:16.209087 19862 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0203 15:03:16.219804 19862 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0203 15:03:16.219878 19862 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0203 15:03:16.227956 19862 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0203 15:03:16.227990 19862 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0203 15:03:16.276561 19862 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0203 15:03:16.276635 19862 kubeadm.go:322] [preflight] Running pre-flight checks
I0203 15:03:16.578090 19862 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0203 15:03:16.578181 19862 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0203 15:03:16.578273 19862 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0203 15:03:16.824121 19862 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0203 15:03:16.824905 19862 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0203 15:03:16.832036 19862 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0203 15:03:16.895111 19862 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0203 15:03:16.917255 19862 out.go:204] - Generating certificates and keys ...
I0203 15:03:16.917345 19862 kubeadm.go:322] [certs] Using existing ca certificate authority
I0203 15:03:16.917397 19862 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0203 15:03:16.917473 19862 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0203 15:03:16.917535 19862 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0203 15:03:16.917602 19862 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0203 15:03:16.917666 19862 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0203 15:03:16.917726 19862 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0203 15:03:16.917828 19862 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0203 15:03:16.917919 19862 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0203 15:03:16.917992 19862 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0203 15:03:16.918020 19862 kubeadm.go:322] [certs] Using the existing "sa" key
I0203 15:03:16.918058 19862 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0203 15:03:17.005004 19862 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0203 15:03:17.192175 19862 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0203 15:03:17.357187 19862 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0203 15:03:17.486700 19862 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0203 15:03:17.487263 19862 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0203 15:03:17.509528 19862 out.go:204] - Booting up control plane ...
I0203 15:03:17.509713 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0203 15:03:17.509846 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0203 15:03:17.509999 19862 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0203 15:03:17.510138 19862 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0203 15:03:17.510451 19862 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0203 15:03:57.496901 19862 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0203 15:03:57.497462 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:03:57.497676 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:04:02.498692 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:04:02.498918 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:04:12.500447 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:04:12.500671 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:04:32.502634 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:04:32.502878 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:05:12.505084 19862 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0203 15:05:12.505243 19862 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0203 15:05:12.505254 19862 kubeadm.go:322]
I0203 15:05:12.505305 19862 kubeadm.go:322] Unfortunately, an error has occurred:
I0203 15:05:12.505357 19862 kubeadm.go:322] timed out waiting for the condition
I0203 15:05:12.505364 19862 kubeadm.go:322]
I0203 15:05:12.505398 19862 kubeadm.go:322] This error is likely caused by:
I0203 15:05:12.505442 19862 kubeadm.go:322] - The kubelet is not running
I0203 15:05:12.505552 19862 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0203 15:05:12.505561 19862 kubeadm.go:322]
I0203 15:05:12.505636 19862 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0203 15:05:12.505658 19862 kubeadm.go:322] - 'systemctl status kubelet'
I0203 15:05:12.505682 19862 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0203 15:05:12.505687 19862 kubeadm.go:322]
I0203 15:05:12.505772 19862 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0203 15:05:12.505842 19862 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0203 15:05:12.505922 19862 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0203 15:05:12.505959 19862 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0203 15:05:12.506018 19862 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0203 15:05:12.506047 19862 kubeadm.go:322] - 'docker logs CONTAINERID'
I0203 15:05:12.508627 19862 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0203 15:05:12.508703 19862 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0203 15:05:12.508814 19862 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
I0203 15:05:12.508908 19862 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0203 15:05:12.508997 19862 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0203 15:05:12.509072 19862 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0203 15:05:12.509092 19862 kubeadm.go:403] StartCluster complete in 3m54.558739312s
I0203 15:05:12.509190 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0203 15:05:12.531869 19862 logs.go:279] 0 containers: []
W0203 15:05:12.531882 19862 logs.go:281] No container was found matching "kube-apiserver"
I0203 15:05:12.531950 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0203 15:05:12.555067 19862 logs.go:279] 0 containers: []
W0203 15:05:12.555079 19862 logs.go:281] No container was found matching "etcd"
I0203 15:05:12.555147 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0203 15:05:12.578514 19862 logs.go:279] 0 containers: []
W0203 15:05:12.578533 19862 logs.go:281] No container was found matching "coredns"
I0203 15:05:12.578599 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0203 15:05:12.602409 19862 logs.go:279] 0 containers: []
W0203 15:05:12.602432 19862 logs.go:281] No container was found matching "kube-scheduler"
I0203 15:05:12.602510 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0203 15:05:12.626377 19862 logs.go:279] 0 containers: []
W0203 15:05:12.626394 19862 logs.go:281] No container was found matching "kube-proxy"
I0203 15:05:12.626485 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0203 15:05:12.649203 19862 logs.go:279] 0 containers: []
W0203 15:05:12.649216 19862 logs.go:281] No container was found matching "kubernetes-dashboard"
I0203 15:05:12.649296 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0203 15:05:12.691137 19862 logs.go:279] 0 containers: []
W0203 15:05:12.691150 19862 logs.go:281] No container was found matching "storage-provisioner"
I0203 15:05:12.691220 19862 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0203 15:05:12.716772 19862 logs.go:279] 0 containers: []
W0203 15:05:12.716786 19862 logs.go:281] No container was found matching "kube-controller-manager"
I0203 15:05:12.716793 19862 logs.go:124] Gathering logs for dmesg ...
I0203 15:05:12.716802 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0203 15:05:12.728969 19862 logs.go:124] Gathering logs for describe nodes ...
I0203 15:05:12.728982 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0203 15:05:12.783506 19862 logs.go:131] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0203 15:05:12.783519 19862 logs.go:124] Gathering logs for Docker ...
I0203 15:05:12.783527 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0203 15:05:12.800613 19862 logs.go:124] Gathering logs for container status ...
I0203 15:05:12.800627 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0203 15:05:14.852190 19862 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.05150261s)
I0203 15:05:14.852319 19862 logs.go:124] Gathering logs for kubelet ...
I0203 15:05:14.852326 19862 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0203 15:05:14.890034 19862 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0203 15:05:14.890060 19862 out.go:239] *
*
W0203 15:05:14.890189 19862 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0203 15:05:14.890207 19862 out.go:239] *
*
W0203 15:05:14.890923 19862 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0203 15:05:14.953455 19862 out.go:177]
W0203 15:05:14.995683 19862 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.23. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0203 15:05:14.995790 19862 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0203 15:05:14.995836 19862 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0203 15:05:15.058641 19862 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-136000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-136000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-136000:
-- stdout --
[
{
"Id": "845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423",
"Created": "2023-02-03T23:01:11.889189264Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 280897,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-02-03T23:01:12.185155252Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:5f59734230331367fdba579a7224885a8ca1b2b3a1b0a3db04074b5e8b329b90",
"ResolvConfPath": "/var/lib/docker/containers/845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423/hostname",
"HostsPath": "/var/lib/docker/containers/845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423/hosts",
"LogPath": "/var/lib/docker/containers/845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423/845795d4cf37caeef2ebc39507d52b464cb71df8ed223e86fa4ff055f8487423-json.log",
"Name": "/old-k8s-version-136000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-136000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-136000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/a8fab6906b656bcd6c37bac3122f87989b3f1a374377d9b548832f7a05b7f2d5-init/diff:/var/lib/docker/overlay2/48b9eff26e94f4439154aad348135bd66f3f3733ee1f2bd22fc60e3a240f764f/diff:/var/lib/docker/overlay2/89930e70b646c5893dab0f6f4274a9fb3b60a11d62da2f59d4b55fbf1c480a90/diff:/var/lib/docker/overlay2/3ae0575a256264d050211e3ca122b2804683b9f4323f7a2c2a2d45f4df3254dd/diff:/var/lib/docker/overlay2/6468a293a6ba199c732872fb7807de809fa2ff9ecdccaeb7146f28e1a4dc9607/diff:/var/lib/docker/overlay2/3fab248b5834a764e1996b2fea0af0100ffc2c150728124745a8e42d43a2193d/diff:/var/lib/docker/overlay2/1ec21b4015d44918fda148d959030dadcaa3527172fde96571978bdabab6921e/diff:/var/lib/docker/overlay2/5465a266a0268ad0ffa1c12afbc320e2232b025ee4eaa5c74b2f5b236ce5285d/diff:/var/lib/docker/overlay2/61b7474b98e6431b966662b98c31f46eb982bdd7098bfccdad928e6c3c0a9024/diff:/var/lib/docker/overlay2/d0925bff8df24b32d176f1438969c0c3adac5ec1bc1da61c2a8bf17e4fd9313b/diff:/var/lib/docker/overlay2/b6c213
617f12dea208efc9c642db1147a22658b32383a0256106a994fcafebca/diff:/var/lib/docker/overlay2/5127e35d4cf68de9ece51806ff390f9b88bac61eaa8bfdf4cf5d6ab1e5b2ca27/diff:/var/lib/docker/overlay2/3d041d254d21e7ec2e2abdce56a3e6eadb3f668238bf3667e7c25effdcc05940/diff:/var/lib/docker/overlay2/15bab989d641601a640d89b58f645e79668cb801bf10066ecd9790e4c8bbd4f1/diff:/var/lib/docker/overlay2/d6e45696a59c84a5b4ad5ad0bec8b561335a71b3c4eaaa35bcbcc00bd3fbcc1a/diff:/var/lib/docker/overlay2/d0a13d3859926a84eb9c7b571fa8c670d15ebf0ab75e6e8971a7b8679b316ca1/diff:/var/lib/docker/overlay2/a5096e1509a8455c4d67f60b17102a08c795ad1bdbeeac3dd75c3b05ec6d922c/diff:/var/lib/docker/overlay2/aeeda7f653d5dcfbb5ef8a7b53a6aba12a5892c04d984f10a71be11833addb2d/diff:/var/lib/docker/overlay2/84bf768303dfde933d5690feb659b1acd5419ca63d78c4760218d578794c3bbe/diff:/var/lib/docker/overlay2/dec6762f77828143e0cb548cc3a6bb9cc10b9f4376070bc49558da8dfd0b7d2e/diff:/var/lib/docker/overlay2/cc9805f6c705d4d0c6c7675e7745ab0dcdd90879809a2089256c0606e80cee7a/diff:/var/lib/d
ocker/overlay2/e34b4063934c19fe1e614a10ef1e9582f55283fa37c9d0b89d0df8ca32a8a03a/diff:/var/lib/docker/overlay2/c6b6cf801ae9739234022d5e5c55176ee1249b3441400f8b9dbde2c15c6d66e3/diff:/var/lib/docker/overlay2/73dfe58a9f4125f321d10ef97d5c2d4951480455bb243f166600ead63c22f5c2/diff:/var/lib/docker/overlay2/476ba412f9e61cc020124b5051db9c99ea08176881e535e0b5fe6ddb51b94a72/diff:/var/lib/docker/overlay2/2729a4e84f2d55dc49c9417254fc26c0baa21f93cd9b58386f869cf5add162c1/diff:/var/lib/docker/overlay2/8523001ce06172b58b31ebf311f62bf435ed3a3d48fec58d3f1239f29386a28b/diff:/var/lib/docker/overlay2/2b7edb3177897200229f3ba188cfd00e16df91cf85b91a5f08ddbfa15d898a3d/diff:/var/lib/docker/overlay2/94231ff2ac5bf304d3c25d204f1a7b2195ef2230bfbb7bb5a1a1d6f2f4faad6a/diff:/var/lib/docker/overlay2/698d3cd800bae40e0aeb942360c67b793550c24bab66ba43080cbcaa500a9069/diff:/var/lib/docker/overlay2/6aadd46423b70866f00e0f4f83310711c1bc22b4dc8989e6b58cd6254540c428/diff:/var/lib/docker/overlay2/035afbe91bfd3bebd444b29f3ceed1e954aab275fca0c8aaf2364df71f4
6e0c3/diff:/var/lib/docker/overlay2/bc68049ba1568fe8bb188720c62bcc993e62a364901ba41a533aa2991cceaf82/diff:/var/lib/docker/overlay2/c3373595ff40ba0ece2698f99fc2e1c9a83c0ef6a1df119125e3009256dee2ed/diff:/var/lib/docker/overlay2/59c87dca7d8987a7e1b5cd959772e06b96d6ecb36399ff9e35a1ecfe4ed33345/diff:/var/lib/docker/overlay2/22434c33a4994657a469b040789f269ac912f4046d76f2531dff05de4700fb3b/diff:/var/lib/docker/overlay2/699ea76dd0a43fedc031501535714f087d7ec3f37593390c9e81c029373c7f8f/diff:/var/lib/docker/overlay2/e9414c264977801651ed9f3ee268cd0f245614747e184e8f3170e1e95d1fc081/diff:/var/lib/docker/overlay2/2781a0c689754699793aa9bdfeeabdaa1c6905e265302dd267c6c12daa01eb9c/diff:/var/lib/docker/overlay2/4b59a1fc73d3e865eaf7e2e62fd6d2808234c79d79b6b30f6b1a482a291580d3/diff:/var/lib/docker/overlay2/7f51e83dcff3227064daa2b7cc6a7c87f8f5e415fa8723316c24512d6029941d/diff:/var/lib/docker/overlay2/50662c60babc4d383f2af76fc66f3712bcc9e85a50f0525fa680c8336af46ce3/diff:/var/lib/docker/overlay2/2112d8437fae31ae95f85bdf08e3f29d09d7b8
adf34c9608a2e3bfecc049e0c0/diff",
"MergedDir": "/var/lib/docker/overlay2/a8fab6906b656bcd6c37bac3122f87989b3f1a374377d9b548832f7a05b7f2d5/merged",
"UpperDir": "/var/lib/docker/overlay2/a8fab6906b656bcd6c37bac3122f87989b3f1a374377d9b548832f7a05b7f2d5/diff",
"WorkDir": "/var/lib/docker/overlay2/a8fab6906b656bcd6c37bac3122f87989b3f1a374377d9b548832f7a05b7f2d5/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-136000",
"Source": "/var/lib/docker/volumes/old-k8s-version-136000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-136000",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.37-1675280603-15763@sha256:9f474b7ba8542a6ea1d4410955102c8c63c61d74579375db5b45bbc427946de8",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-136000",
"name.minikube.sigs.k8s.io": "old-k8s-version-136000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "13c5a8c73f7d0b11fc92b00e1930aa2a554af42a7b3351904954459cbd927fce",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55132"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55133"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55134"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55135"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "55136"
}
]
},
"SandboxKey": "/var/run/docker/netns/13c5a8c73f7d",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-136000": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"845795d4cf37",
"old-k8s-version-136000"
],
"NetworkID": "a4c82c2a3592223db620bf95332091613324019646bbe58152af123c5085aba4",
"EndpointID": "edd09a73a6c2e6bd1a1f9a964b90c22bb2999232e698e5ae8e29125bd541e10e",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-136000 -n old-k8s-version-136000
E0203 15:05:15.580652 2568 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/15770-1719/.minikube/profiles/calico-292000/client.crt: no such file or directory
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-136000 -n old-k8s-version-136000: exit status 6 (420.679574ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0203 15:05:15.629076 20961 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-136000" does not appear in /Users/jenkins/minikube-integration/15770-1719/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-136000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (251.78s)