=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-20220630125818-3890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
=== CONT TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-20220630125818-3890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m10.956253008s)
-- stdout --
* [old-k8s-version-20220630125818-3890] minikube v1.26.0 on Darwin 12.4
- MINIKUBE_LOCATION=14420
- KUBECONFIG=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-20220630125818-3890 in cluster old-k8s-version-20220630125818-3890
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 20.10.17 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0630 12:58:18.573181 18674 out.go:296] Setting OutFile to fd 1 ...
I0630 12:58:18.573385 18674 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 12:58:18.573390 18674 out.go:309] Setting ErrFile to fd 2...
I0630 12:58:18.573394 18674 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 12:58:18.573714 18674 root.go:329] Updating PATH: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/bin
I0630 12:58:18.574016 18674 out.go:303] Setting JSON to false
I0630 12:58:18.590456 18674 start.go:115] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":5271,"bootTime":1656613827,"procs":349,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"12.4","kernelVersion":"21.5.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}
W0630 12:58:18.590563 18674 start.go:123] gopshost.Virtualization returned error: not implemented yet
I0630 12:58:18.632687 18674 out.go:177] * [old-k8s-version-20220630125818-3890] minikube v1.26.0 on Darwin 12.4
I0630 12:58:18.676684 18674 notify.go:193] Checking for updates...
I0630 12:58:18.719410 18674 out.go:177] - MINIKUBE_LOCATION=14420
I0630 12:58:18.763679 18674 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/kubeconfig
I0630 12:58:18.843800 18674 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0630 12:58:18.886801 18674 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0630 12:58:18.950938 18674 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube
I0630 12:58:19.010303 18674 config.go:178] Loaded profile config "kubenet-20220630124055-3890": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.24.2
I0630 12:58:19.010397 18674 driver.go:360] Setting default libvirt URI to qemu:///system
I0630 12:58:19.160475 18674 docker.go:137] docker version: linux-20.10.17
I0630 12:58:19.160602 18674 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 12:58:19.336984 18674 info.go:265] docker info: {ID:E5HC:V2LT:KNLT:54RE:DWJM:FAYV:KYJI:HFXY:FV7S:EBZT:ZENB:ZHQK Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:57 OomKillDisable:false NGoroutines:51 SystemTime:2022-06-30 19:58:19.256847719 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:3 KernelVersion:5.10.104-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexSer
verAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6232580096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defau
lt name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/local/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2] map[Name:compose Path:/usr/local/lib/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.1] map[Name:extension Path:/usr/local/lib/docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.7] map[Name:sbom Path:/usr/local/lib/docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/usr/local/lib/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 12:58:19.400684 18674 out.go:177] * Using the docker driver based on user configuration
I0630 12:58:19.422578 18674 start.go:284] selected driver: docker
I0630 12:58:19.422644 18674 start.go:808] validating driver "docker" against <nil>
I0630 12:58:19.422671 18674 start.go:819] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0630 12:58:19.425942 18674 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 12:58:19.630524 18674 info.go:265] docker info: {ID:E5HC:V2LT:KNLT:54RE:DWJM:FAYV:KYJI:HFXY:FV7S:EBZT:ZENB:ZHQK Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:57 OomKillDisable:false NGoroutines:51 SystemTime:2022-06-30 19:58:19.529767537 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:3 KernelVersion:5.10.104-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexSer
verAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6232580096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=defau
lt name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/local/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2] map[Name:compose Path:/usr/local/lib/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.1] map[Name:extension Path:/usr/local/lib/docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.7] map[Name:sbom Path:/usr/local/lib/docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/usr/local/lib/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 12:58:19.630670 18674 start_flags.go:296] no existing cluster config was found, will generate one from the flags
I0630 12:58:19.630859 18674 start_flags.go:853] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0630 12:58:19.671711 18674 out.go:177] * Using Docker Desktop driver with root privileges
I0630 12:58:19.735594 18674 cni.go:95] Creating CNI manager for ""
I0630 12:58:19.735620 18674 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0630 12:58:19.735631 18674 start_flags.go:310] config:
{Name:old-k8s-version-20220630125818-3890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-20220630125818-3890 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDom
ain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 12:58:19.778832 18674 out.go:177] * Starting control plane node old-k8s-version-20220630125818-3890 in cluster old-k8s-version-20220630125818-3890
I0630 12:58:19.820541 18674 cache.go:120] Beginning downloading kic base image for docker with docker
I0630 12:58:19.842823 18674 out.go:177] * Pulling base image ...
I0630 12:58:19.886666 18674 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0630 12:58:19.886677 18674 image.go:75] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon
I0630 12:58:19.886749 18674 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0630 12:58:19.886773 18674 cache.go:57] Caching tarball of preloaded images
I0630 12:58:19.887759 18674 preload.go:174] Found /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0630 12:58:19.887858 18674 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0630 12:58:19.888252 18674 profile.go:148] Saving config to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/config.json ...
I0630 12:58:19.888318 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/config.json: {Name:mk09ea0ced3d0b217ceb3628e3d22e4652a00676 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:20.040984 18674 image.go:79] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon, skipping pull
I0630 12:58:20.041010 18674 cache.go:142] gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e exists in daemon, skipping load
I0630 12:58:20.041023 18674 cache.go:208] Successfully downloaded all kic artifacts
I0630 12:58:20.041089 18674 start.go:352] acquiring machines lock for old-k8s-version-20220630125818-3890: {Name:mk7eef66d725580b285e0376127b4623b8f07ae2 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0630 12:58:20.060902 18674 start.go:356] acquired machines lock for "old-k8s-version-20220630125818-3890" in 19.781766ms
I0630 12:58:20.060998 18674 start.go:91] Provisioning new machine with config: &{Name:old-k8s-version-20220630125818-3890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-20220630125818-3890 N
amespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirr
or: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0630 12:58:20.061149 18674 start.go:131] createHost starting for "" (driver="docker")
I0630 12:58:20.105632 18674 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0630 12:58:20.106214 18674 start.go:165] libmachine.API.Create for "old-k8s-version-20220630125818-3890" (driver="docker")
I0630 12:58:20.106278 18674 client.go:168] LocalClient.Create starting
I0630 12:58:20.106451 18674 main.go:134] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem
I0630 12:58:20.106562 18674 main.go:134] libmachine: Decoding PEM data...
I0630 12:58:20.106611 18674 main.go:134] libmachine: Parsing certificate...
I0630 12:58:20.106777 18674 main.go:134] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/cert.pem
I0630 12:58:20.106856 18674 main.go:134] libmachine: Decoding PEM data...
I0630 12:58:20.106891 18674 main.go:134] libmachine: Parsing certificate...
I0630 12:58:20.128842 18674 cli_runner.go:164] Run: docker network inspect old-k8s-version-20220630125818-3890 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0630 12:58:20.286821 18674 cli_runner.go:211] docker network inspect old-k8s-version-20220630125818-3890 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0630 12:58:20.286917 18674 network_create.go:272] running [docker network inspect old-k8s-version-20220630125818-3890] to gather additional debugging logs...
I0630 12:58:20.286967 18674 cli_runner.go:164] Run: docker network inspect old-k8s-version-20220630125818-3890
W0630 12:58:20.469978 18674 cli_runner.go:211] docker network inspect old-k8s-version-20220630125818-3890 returned with exit code 1
I0630 12:58:20.470036 18674 network_create.go:275] error running [docker network inspect old-k8s-version-20220630125818-3890]: docker network inspect old-k8s-version-20220630125818-3890: exit status 1
stdout:
[]
stderr:
Error: No such network: old-k8s-version-20220630125818-3890
I0630 12:58:20.470057 18674 network_create.go:277] output of [docker network inspect old-k8s-version-20220630125818-3890]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: old-k8s-version-20220630125818-3890
** /stderr **
I0630 12:58:20.470161 18674 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0630 12:58:20.615279 18674 network.go:288] reserving subnet 192.168.49.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.49.0:0xc00064a520] misses:0}
I0630 12:58:20.615321 18674 network.go:235] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.615339 18674 network_create.go:115] attempt to create docker network old-k8s-version-20220630125818-3890 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 1500 ...
I0630 12:58:20.615402 18674 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890
W0630 12:58:20.698547 18674 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890 returned with exit code 1
W0630 12:58:20.698598 18674 network_create.go:107] failed to create docker network old-k8s-version-20220630125818-3890 192.168.49.0/24, will retry: subnet is taken
I0630 12:58:20.698903 18674 network.go:279] skipping subnet 192.168.49.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:false}} dirty:map[] misses:0}
I0630 12:58:20.698922 18674 network.go:238] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.699128 18674 network.go:288] reserving subnet 192.168.58.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:true}} dirty:map[192.168.49.0:0xc00064a520 192.168.58.0:0xc000e602a0] misses:0}
I0630 12:58:20.699140 18674 network.go:235] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.699147 18674 network_create.go:115] attempt to create docker network old-k8s-version-20220630125818-3890 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 1500 ...
I0630 12:58:20.699207 18674 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890
W0630 12:58:20.764475 18674 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890 returned with exit code 1
W0630 12:58:20.764512 18674 network_create.go:107] failed to create docker network old-k8s-version-20220630125818-3890 192.168.58.0/24, will retry: subnet is taken
I0630 12:58:20.764782 18674 network.go:279] skipping subnet 192.168.58.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:true}} dirty:map[192.168.49.0:0xc00064a520 192.168.58.0:0xc000e602a0] misses:1}
I0630 12:58:20.764801 18674 network.go:238] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.765000 18674 network.go:288] reserving subnet 192.168.67.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:true}} dirty:map[192.168.49.0:0xc00064a520 192.168.58.0:0xc000e602a0 192.168.67.0:0xc0006de168] misses:1}
I0630 12:58:20.765016 18674 network.go:235] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.765023 18674 network_create.go:115] attempt to create docker network old-k8s-version-20220630125818-3890 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 1500 ...
I0630 12:58:20.765080 18674 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890
W0630 12:58:20.828346 18674 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890 returned with exit code 1
W0630 12:58:20.828387 18674 network_create.go:107] failed to create docker network old-k8s-version-20220630125818-3890 192.168.67.0/24, will retry: subnet is taken
I0630 12:58:20.828649 18674 network.go:279] skipping subnet 192.168.67.0 that has unexpired reservation: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:true}} dirty:map[192.168.49.0:0xc00064a520 192.168.58.0:0xc000e602a0 192.168.67.0:0xc0006de168] misses:2}
I0630 12:58:20.828668 18674 network.go:238] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.828865 18674 network.go:288] reserving subnet 192.168.76.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[192.168.49.0:0xc00064a520] amended:true}} dirty:map[192.168.49.0:0xc00064a520 192.168.58.0:0xc000e602a0 192.168.67.0:0xc0006de168 192.168.76.0:0xc000e602d8] misses:2}
I0630 12:58:20.828877 18674 network.go:235] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 12:58:20.828883 18674 network_create.go:115] attempt to create docker network old-k8s-version-20220630125818-3890 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0630 12:58:20.828935 18674 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 old-k8s-version-20220630125818-3890
I0630 12:58:20.924701 18674 network_create.go:99] docker network old-k8s-version-20220630125818-3890 192.168.76.0/24 created
I0630 12:58:20.924732 18674 kic.go:106] calculated static IP "192.168.76.2" for the "old-k8s-version-20220630125818-3890" container
I0630 12:58:20.924825 18674 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0630 12:58:20.994404 18674 cli_runner.go:164] Run: docker volume create old-k8s-version-20220630125818-3890 --label name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 --label created_by.minikube.sigs.k8s.io=true
I0630 12:58:21.066768 18674 oci.go:103] Successfully created a docker volume old-k8s-version-20220630125818-3890
I0630 12:58:21.066888 18674 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-20220630125818-3890-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 --entrypoint /usr/bin/test -v old-k8s-version-20220630125818-3890:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -d /var/lib
I0630 12:58:21.564702 18674 oci.go:107] Successfully prepared a docker volume old-k8s-version-20220630125818-3890
I0630 12:58:21.564740 18674 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0630 12:58:21.564753 18674 kic.go:179] Starting extracting preloaded images to volume ...
I0630 12:58:21.564846 18674 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-20220630125818-3890:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -I lz4 -xf /preloaded.tar -C /extractDir
I0630 12:58:25.725574 18674 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-20220630125818-3890:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -I lz4 -xf /preloaded.tar -C /extractDir: (4.160591903s)
I0630 12:58:25.725594 18674 kic.go:188] duration metric: took 4.160808 seconds to extract preloaded images to volume
I0630 12:58:25.725682 18674 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0630 12:58:25.897680 18674 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-20220630125818-3890 --name old-k8s-version-20220630125818-3890 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-20220630125818-3890 --network old-k8s-version-20220630125818-3890 --ip 192.168.76.2 --volume old-k8s-version-20220630125818-3890:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=8443 --publish=22 --publish=2376 --publish=5000 --publish=32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e
I0630 12:58:26.367707 18674 cli_runner.go:164] Run: docker container inspect old-k8s-version-20220630125818-3890 --format={{.State.Running}}
I0630 12:58:26.450591 18674 cli_runner.go:164] Run: docker container inspect old-k8s-version-20220630125818-3890 --format={{.State.Status}}
I0630 12:58:26.532557 18674 cli_runner.go:164] Run: docker exec old-k8s-version-20220630125818-3890 stat /var/lib/dpkg/alternatives/iptables
I0630 12:58:26.671045 18674 oci.go:144] the created container "old-k8s-version-20220630125818-3890" has a running status.
I0630 12:58:26.671074 18674 kic.go:210] Creating ssh key for kic: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa...
I0630 12:58:26.798463 18674 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0630 12:58:26.943816 18674 cli_runner.go:164] Run: docker container inspect old-k8s-version-20220630125818-3890 --format={{.State.Status}}
I0630 12:58:27.024601 18674 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0630 12:58:27.024621 18674 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-20220630125818-3890 chown docker:docker /home/docker/.ssh/authorized_keys]
I0630 12:58:27.159371 18674 cli_runner.go:164] Run: docker container inspect old-k8s-version-20220630125818-3890 --format={{.State.Status}}
I0630 12:58:27.237914 18674 machine.go:88] provisioning docker machine ...
I0630 12:58:27.237956 18674 ubuntu.go:169] provisioning hostname "old-k8s-version-20220630125818-3890"
I0630 12:58:27.238067 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:27.319058 18674 main.go:134] libmachine: Using SSH client type: native
I0630 12:58:27.319240 18674 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13d2d60] 0x13d5dc0 <nil> [] 0s} 127.0.0.1 61111 <nil> <nil>}
I0630 12:58:27.319263 18674 main.go:134] libmachine: About to run SSH command:
sudo hostname old-k8s-version-20220630125818-3890 && echo "old-k8s-version-20220630125818-3890" | sudo tee /etc/hostname
I0630 12:58:27.455031 18674 main.go:134] libmachine: SSH cmd err, output: <nil>: old-k8s-version-20220630125818-3890
I0630 12:58:27.455118 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:27.570760 18674 main.go:134] libmachine: Using SSH client type: native
I0630 12:58:27.570912 18674 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13d2d60] 0x13d5dc0 <nil> [] 0s} 127.0.0.1 61111 <nil> <nil>}
I0630 12:58:27.570934 18674 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-20220630125818-3890' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-20220630125818-3890/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-20220630125818-3890' | sudo tee -a /etc/hosts;
fi
fi
I0630 12:58:27.687328 18674 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0630 12:58:27.687354 18674 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube CaCertPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/se
rver.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube}
I0630 12:58:27.687386 18674 ubuntu.go:177] setting up certificates
I0630 12:58:27.687399 18674 provision.go:83] configureAuth start
I0630 12:58:27.687476 18674 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-20220630125818-3890
I0630 12:58:27.761024 18674 provision.go:138] copyHostCerts
I0630 12:58:27.761119 18674 exec_runner.go:144] found /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.pem, removing ...
I0630 12:58:27.761152 18674 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.pem
I0630 12:58:27.761256 18674 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.pem (1082 bytes)
I0630 12:58:27.761476 18674 exec_runner.go:144] found /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cert.pem, removing ...
I0630 12:58:27.761495 18674 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cert.pem
I0630 12:58:27.761557 18674 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/cert.pem (1123 bytes)
I0630 12:58:27.761711 18674 exec_runner.go:144] found /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/key.pem, removing ...
I0630 12:58:27.761717 18674 exec_runner.go:207] rm: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/key.pem
I0630 12:58:27.761784 18674 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/key.pem (1679 bytes)
I0630 12:58:27.761898 18674 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-20220630125818-3890 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-20220630125818-3890]
I0630 12:58:27.919022 18674 provision.go:172] copyRemoteCerts
I0630 12:58:27.919357 18674 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0630 12:58:27.919411 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:27.996034 18674 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61111 SSHKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa Username:docker}
I0630 12:58:28.083109 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0630 12:58:28.100376 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/server.pem --> /etc/docker/server.pem (1277 bytes)
I0630 12:58:28.117468 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0630 12:58:28.134216 18674 provision.go:86] duration metric: configureAuth took 446.802328ms
I0630 12:58:28.134245 18674 ubuntu.go:193] setting minikube options for container-runtime
I0630 12:58:28.134417 18674 config.go:178] Loaded profile config "old-k8s-version-20220630125818-3890": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0630 12:58:28.134555 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:28.208597 18674 main.go:134] libmachine: Using SSH client type: native
I0630 12:58:28.208816 18674 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13d2d60] 0x13d5dc0 <nil> [] 0s} 127.0.0.1 61111 <nil> <nil>}
I0630 12:58:28.208856 18674 main.go:134] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0630 12:58:28.327213 18674 main.go:134] libmachine: SSH cmd err, output: <nil>: overlay
I0630 12:58:28.327229 18674 ubuntu.go:71] root file system type: overlay
I0630 12:58:28.327336 18674 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0630 12:58:28.327404 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:28.402713 18674 main.go:134] libmachine: Using SSH client type: native
I0630 12:58:28.402871 18674 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13d2d60] 0x13d5dc0 <nil> [] 0s} 127.0.0.1 61111 <nil> <nil>}
I0630 12:58:28.402952 18674 main.go:134] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0630 12:58:28.531892 18674 main.go:134] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0630 12:58:28.531987 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:28.611025 18674 main.go:134] libmachine: Using SSH client type: native
I0630 12:58:28.611206 18674 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x13d2d60] 0x13d5dc0 <nil> [] 0s} 127.0.0.1 61111 <nil> <nil>}
I0630 12:58:28.611219 18674 main.go:134] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0630 12:58:29.210696 18674 main.go:134] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2022-06-06 23:01:03.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2022-06-30 19:58:28.550105207 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0630 12:58:29.210718 18674 machine.go:91] provisioned docker machine in 1.972770425s
I0630 12:58:29.210726 18674 client.go:171] LocalClient.Create took 9.104372836s
I0630 12:58:29.210741 18674 start.go:173] duration metric: libmachine.API.Create for "old-k8s-version-20220630125818-3890" took 9.104462627s
I0630 12:58:29.210749 18674 start.go:306] post-start starting for "old-k8s-version-20220630125818-3890" (driver="docker")
I0630 12:58:29.210753 18674 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0630 12:58:29.210852 18674 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0630 12:58:29.210903 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:29.285704 18674 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61111 SSHKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa Username:docker}
I0630 12:58:29.374685 18674 ssh_runner.go:195] Run: cat /etc/os-release
I0630 12:58:29.379999 18674 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0630 12:58:29.380015 18674 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0630 12:58:29.380025 18674 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0630 12:58:29.380031 18674 info.go:137] Remote host: Ubuntu 20.04.4 LTS
I0630 12:58:29.380044 18674 filesync.go:126] Scanning /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/addons for local assets ...
I0630 12:58:29.380163 18674 filesync.go:126] Scanning /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files for local assets ...
I0630 12:58:29.380310 18674 filesync.go:149] local asset: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files/etc/ssl/certs/38902.pem -> 38902.pem in /etc/ssl/certs
I0630 12:58:29.380462 18674 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0630 12:58:29.387866 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files/etc/ssl/certs/38902.pem --> /etc/ssl/certs/38902.pem (1708 bytes)
I0630 12:58:29.404412 18674 start.go:309] post-start completed in 193.653851ms
I0630 12:58:29.404956 18674 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-20220630125818-3890
I0630 12:58:29.479202 18674 profile.go:148] Saving config to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/config.json ...
I0630 12:58:29.479655 18674 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0630 12:58:29.479706 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:29.552881 18674 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61111 SSHKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa Username:docker}
I0630 12:58:29.637675 18674 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0630 12:58:29.642008 18674 start.go:134] duration metric: createHost completed in 9.580777733s
I0630 12:58:29.642025 18674 start.go:81] releasing machines lock for "old-k8s-version-20220630125818-3890", held for 9.581006564s
I0630 12:58:29.642105 18674 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-20220630125818-3890
I0630 12:58:29.714557 18674 ssh_runner.go:195] Run: systemctl --version
I0630 12:58:29.714558 18674 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0630 12:58:29.714647 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:29.714657 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:29.796425 18674 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61111 SSHKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa Username:docker}
I0630 12:58:29.798585 18674 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:61111 SSHKeyPath:/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/machines/old-k8s-version-20220630125818-3890/id_rsa Username:docker}
I0630 12:58:29.881843 18674 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0630 12:58:30.347860 18674 cruntime.go:273] skipping containerd shutdown because we are bound to it
I0630 12:58:30.347945 18674 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0630 12:58:30.357825 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0630 12:58:30.370670 18674 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0630 12:58:30.440513 18674 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0630 12:58:30.512579 18674 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0630 12:58:30.576589 18674 ssh_runner.go:195] Run: sudo systemctl restart docker
I0630 12:58:30.781905 18674 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0630 12:58:30.816944 18674 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0630 12:58:30.875122 18674 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 20.10.17 ...
I0630 12:58:30.875266 18674 cli_runner.go:164] Run: docker exec -t old-k8s-version-20220630125818-3890 dig +short host.docker.internal
I0630 12:58:31.004321 18674 network.go:96] got host ip for mount in container by digging dns: 192.168.65.2
I0630 12:58:31.005072 18674 ssh_runner.go:195] Run: grep 192.168.65.2 host.minikube.internal$ /etc/hosts
I0630 12:58:31.009058 18674 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 12:58:31.022103 18674 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-20220630125818-3890
I0630 12:58:31.096871 18674 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0630 12:58:31.096932 18674 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0630 12:58:31.127107 18674 docker.go:602] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0630 12:58:31.127124 18674 docker.go:533] Images already preloaded, skipping extraction
I0630 12:58:31.127215 18674 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0630 12:58:31.156343 18674 docker.go:602] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0630 12:58:31.156362 18674 cache_images.go:84] Images are preloaded, skipping loading
I0630 12:58:31.156445 18674 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0630 12:58:31.230007 18674 cni.go:95] Creating CNI manager for ""
I0630 12:58:31.230021 18674 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0630 12:58:31.230043 18674 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0630 12:58:31.230060 18674 kubeadm.go:158] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-20220630125818-3890 NodeName:old-k8s-version-20220630125818-3890 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.76.2 CgroupDriver:systemd ClientC
AFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0630 12:58:31.230232 18674 kubeadm.go:162] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-20220630125818-3890"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-20220630125818-3890
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.76.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: systemd
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0630 12:58:31.230308 18674 kubeadm.go:961] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-20220630125818-3890 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-20220630125818-3890 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0630 12:58:31.230371 18674 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0630 12:58:31.238063 18674 binaries.go:44] Found k8s binaries, skipping transfer
I0630 12:58:31.238117 18674 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0630 12:58:31.245232 18674 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (361 bytes)
I0630 12:58:31.257931 18674 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0630 12:58:31.270486 18674 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2146 bytes)
I0630 12:58:31.283163 18674 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0630 12:58:31.286958 18674 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 12:58:31.296330 18674 certs.go:54] Setting up /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890 for IP: 192.168.76.2
I0630 12:58:31.296493 18674 certs.go:182] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.key
I0630 12:58:31.296542 18674 certs.go:182] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/proxy-client-ca.key
I0630 12:58:31.296583 18674 certs.go:302] generating minikube-user signed cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.key
I0630 12:58:31.296615 18674 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.crt with IP's: []
I0630 12:58:31.401026 18674 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.crt ...
I0630 12:58:31.401041 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.crt: {Name:mk07a0c9be566eddfaaea92abae478ec4aefebe8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.401365 18674 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.key ...
I0630 12:58:31.401373 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/client.key: {Name:mk142525b9b1b142720946f7e82528638f5c6b3b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.401575 18674 certs.go:302] generating minikube signed cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key.31bdca25
I0630 12:58:31.401592 18674 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0630 12:58:31.547387 18674 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt.31bdca25 ...
I0630 12:58:31.547406 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt.31bdca25: {Name:mk5bbbc2c5a135f90d3daaf53ed2f31616cfd0b8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.547660 18674 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key.31bdca25 ...
I0630 12:58:31.547667 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key.31bdca25: {Name:mkf49a88124982ec94501458e54b9fc912111887 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.547844 18674 certs.go:320] copying /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt.31bdca25 -> /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt
I0630 12:58:31.547990 18674 certs.go:324] copying /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key.31bdca25 -> /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key
I0630 12:58:31.548138 18674 certs.go:302] generating aggregator signed cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.key
I0630 12:58:31.548151 18674 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.crt with IP's: []
I0630 12:58:31.661981 18674 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.crt ...
I0630 12:58:31.661994 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.crt: {Name:mk7311738c41506143eb97b7a32fe2619a04fe7f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.662293 18674 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.key ...
I0630 12:58:31.662300 18674 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.key: {Name:mk3796aef0b4e7222271f9e91c23992b2a2dc73d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 12:58:31.662840 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/3890.pem (1338 bytes)
W0630 12:58:31.662883 18674 certs.go:384] ignoring /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/3890_empty.pem, impossibly tiny 0 bytes
I0630 12:58:31.662892 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca-key.pem (1679 bytes)
I0630 12:58:31.662920 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/ca.pem (1082 bytes)
I0630 12:58:31.662948 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/cert.pem (1123 bytes)
I0630 12:58:31.662976 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/key.pem (1679 bytes)
I0630 12:58:31.663079 18674 certs.go:388] found cert: /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files/etc/ssl/certs/38902.pem (1708 bytes)
I0630 12:58:31.663558 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0630 12:58:31.684630 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0630 12:58:31.701755 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0630 12:58:31.718583 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/old-k8s-version-20220630125818-3890/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0630 12:58:31.735634 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0630 12:58:31.752999 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0630 12:58:31.769937 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0630 12:58:31.788703 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0630 12:58:31.805676 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/files/etc/ssl/certs/38902.pem --> /usr/share/ca-certificates/38902.pem (1708 bytes)
I0630 12:58:31.822729 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0630 12:58:31.840472 18674 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/certs/3890.pem --> /usr/share/ca-certificates/3890.pem (1338 bytes)
I0630 12:58:31.857890 18674 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0630 12:58:31.870682 18674 ssh_runner.go:195] Run: openssl version
I0630 12:58:31.876037 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/38902.pem && ln -fs /usr/share/ca-certificates/38902.pem /etc/ssl/certs/38902.pem"
I0630 12:58:31.883887 18674 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/38902.pem
I0630 12:58:31.887935 18674 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Jun 30 19:04 /usr/share/ca-certificates/38902.pem
I0630 12:58:31.887974 18674 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/38902.pem
I0630 12:58:31.893413 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/38902.pem /etc/ssl/certs/3ec20f2e.0"
I0630 12:58:31.901063 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0630 12:58:31.909055 18674 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0630 12:58:31.912973 18674 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Jun 30 18:59 /usr/share/ca-certificates/minikubeCA.pem
I0630 12:58:31.913039 18674 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0630 12:58:31.918113 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0630 12:58:31.925787 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3890.pem && ln -fs /usr/share/ca-certificates/3890.pem /etc/ssl/certs/3890.pem"
I0630 12:58:31.933827 18674 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3890.pem
I0630 12:58:31.937714 18674 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Jun 30 19:04 /usr/share/ca-certificates/3890.pem
I0630 12:58:31.937758 18674 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3890.pem
I0630 12:58:31.942941 18674 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/3890.pem /etc/ssl/certs/51391683.0"
I0630 12:58:31.950639 18674 kubeadm.go:395] StartCluster: {Name:old-k8s-version-20220630125818-3890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-20220630125818-3890 Namespace:default APISe
rverName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: Disabl
eOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 12:58:31.950728 18674 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0630 12:58:31.979787 18674 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0630 12:58:31.987634 18674 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0630 12:58:31.994945 18674 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 12:58:31.994993 18674 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 12:58:32.002229 18674 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 12:58:32.002255 18674 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 12:58:32.770163 18674 out.go:204] - Generating certificates and keys ...
I0630 12:58:34.947474 18674 out.go:204] - Booting up control plane ...
W0630 13:00:29.871638 18674 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-20220630125818-3890 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-20220630125818-3890 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-20220630125818-3890 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-20220630125818-3890 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0630 13:00:29.871671 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0630 13:00:30.292373 18674 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 13:00:30.301998 18674 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 13:00:30.302045 18674 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 13:00:30.310174 18674 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 13:00:30.310203 18674 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 13:00:31.056191 18674 out.go:204] - Generating certificates and keys ...
I0630 13:00:31.884178 18674 out.go:204] - Booting up control plane ...
I0630 13:02:26.805109 18674 kubeadm.go:397] StartCluster complete in 3m54.852728426s
I0630 13:02:26.805184 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0630 13:02:26.834833 18674 logs.go:274] 0 containers: []
W0630 13:02:26.834844 18674 logs.go:276] No container was found matching "kube-apiserver"
I0630 13:02:26.834902 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0630 13:02:26.864604 18674 logs.go:274] 0 containers: []
W0630 13:02:26.864616 18674 logs.go:276] No container was found matching "etcd"
I0630 13:02:26.864673 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0630 13:02:26.894684 18674 logs.go:274] 0 containers: []
W0630 13:02:26.894696 18674 logs.go:276] No container was found matching "coredns"
I0630 13:02:26.894756 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0630 13:02:26.923834 18674 logs.go:274] 0 containers: []
W0630 13:02:26.923846 18674 logs.go:276] No container was found matching "kube-scheduler"
I0630 13:02:26.923909 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0630 13:02:26.954447 18674 logs.go:274] 0 containers: []
W0630 13:02:26.954461 18674 logs.go:276] No container was found matching "kube-proxy"
I0630 13:02:26.954528 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0630 13:02:27.019368 18674 logs.go:274] 0 containers: []
W0630 13:02:27.019381 18674 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 13:02:27.019436 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0630 13:02:27.052621 18674 logs.go:274] 0 containers: []
W0630 13:02:27.052634 18674 logs.go:276] No container was found matching "storage-provisioner"
I0630 13:02:27.052696 18674 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0630 13:02:27.084258 18674 logs.go:274] 0 containers: []
W0630 13:02:27.084271 18674 logs.go:276] No container was found matching "kube-controller-manager"
I0630 13:02:27.084278 18674 logs.go:123] Gathering logs for describe nodes ...
I0630 13:02:27.084285 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 13:02:27.141010 18674 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 13:02:27.141027 18674 logs.go:123] Gathering logs for Docker ...
I0630 13:02:27.141033 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0630 13:02:27.157642 18674 logs.go:123] Gathering logs for container status ...
I0630 13:02:27.157654 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 13:02:29.215181 18674 ssh_runner.go:235] Completed: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": (2.057499862s)
I0630 13:02:29.215307 18674 logs.go:123] Gathering logs for kubelet ...
I0630 13:02:29.215315 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0630 13:02:29.255583 18674 logs.go:123] Gathering logs for dmesg ...
I0630 13:02:29.255596 18674 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
W0630 13:02:29.268656 18674 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 13:02:29.268684 18674 out.go:239] *
*
W0630 13:02:29.268815 18674 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 13:02:29.268830 18674 out.go:239] *
*
W0630 13:02:29.269381 18674 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0630 13:02:29.331923 18674 out.go:177]
W0630 13:02:29.374158 18674 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.17. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 13:02:29.374228 18674 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0630 13:02:29.374277 18674 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0630 13:02:29.416115 18674 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-20220630125818-3890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-20220630125818-3890
helpers_test.go:235: (dbg) docker inspect old-k8s-version-20220630125818-3890:
-- stdout --
[
{
"Id": "aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc",
"Created": "2022-06-30T19:58:25.991954109Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 231823,
"ExitCode": 0,
"Error": "",
"StartedAt": "2022-06-30T19:58:26.376119053Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:573e7be5768273a7845baee1ae90fa2e33b83b10a7fbb0f0f41efbf29b53d1f1",
"ResolvConfPath": "/var/lib/docker/containers/aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc/hostname",
"HostsPath": "/var/lib/docker/containers/aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc/hosts",
"LogPath": "/var/lib/docker/containers/aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc/aa122ad07739e4ac0fb8bdf9b3e744955c331616e5741f704b3f39f3519bf0bc-json.log",
"Name": "/old-k8s-version-20220630125818-3890",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-20220630125818-3890:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-20220630125818-3890",
"PortBindings": {
"22/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/2b600975cc55b63b3fd9b41576fefb8a21fc4ff6823c2d3736d297507e06a0fb-init/diff:/var/lib/docker/overlay2/ffce70bf4f5c6f1825a937d47f8331eb47d81eb1246e084d39182cc01d6fb8fc/diff:/var/lib/docker/overlay2/726cc53b5195287a64c158dd0221b5e970d274c142a7a1126fecf9fab6c6578f/diff:/var/lib/docker/overlay2/a880e3df20642f54bc012c7ed0b4a206e7e36bd578f8005ab318b29038101ff0/diff:/var/lib/docker/overlay2/417501ed9b591648606834b4e69aebc86ec839cf597a46a6ba206d5d038f7bb1/diff:/var/lib/docker/overlay2/7a1a9ab6170ca917aacb0a788d443d099c6e9eb621a72a5d1308e07d66b3dab5/diff:/var/lib/docker/overlay2/69398fcad2efcd38a4934d8bc015514ec12eb76624f0ccfdf200f2e1de3cd0de/diff:/var/lib/docker/overlay2/ba0049c7c2511af8f50f5533a1533c3d5e2bf9859bffa68237b39630c5123707/diff:/var/lib/docker/overlay2/bc43d7c4f245b957fef59ca08842ecff68172da2046c1c436969174b5ebaf798/diff:/var/lib/docker/overlay2/8b38b041b160228a3f7656e44a1b236afcdb06576ffc0de46c096963aee80d1e/diff:/var/lib/docker/overlay2/077354
c87a507d4adc039d3486a5d7bfa2003d612b2b58debaed501a2476ad53/diff:/var/lib/docker/overlay2/b6d3d6b59f910daf40f8f267e3fb8afc2594ec1ede45ce2b7b3550e212a5c6ba/diff:/var/lib/docker/overlay2/c62d2476238bf9ba408dd809af6c21ec6f988e5910c3b70bdcb9d0610c94b588/diff:/var/lib/docker/overlay2/2a92e2066a09e871a4dd249759bd728f7b0e3e9714f86936a87b6abd2f406eae/diff:/var/lib/docker/overlay2/4d874f52f5def19df5bd1c37fe54f9b14b189a08de4936d8428d47387e68813a/diff:/var/lib/docker/overlay2/90983b61a6951f8a918181d8ee79ff9df959b6e28749a277096a37a27c83474d/diff:/var/lib/docker/overlay2/76c2b7fe336563bed6c3f8af424211170d900eddeaacabccbfd32c5198e49315/diff:/var/lib/docker/overlay2/3114e2dac4d9978b5005b9704e4db04198618585c02ace12fb50ea40885bf9ab/diff:/var/lib/docker/overlay2/8ff489b60f3dd34df6f3f4c58887d692126c64c6fce53e26e52a35e321e05f0c/diff:/var/lib/docker/overlay2/489b6f8e71c6903f7cff82340f1bba363124f09ba9458385b6506c9e5ec2f15f/diff:/var/lib/docker/overlay2/86adec944624d780606c4165061133e21345180a10ccfc61bad7021cc4ee88d1/diff:/var/lib/d
ocker/overlay2/da8a353ae672de9b7126c54ee9eb4d10cb01339f5453aeec978f90cb51bc2847/diff:/var/lib/docker/overlay2/9c229a55ca048d684fd84a74267d4934b7dd3451591b438ab12182780194bec5/diff:/var/lib/docker/overlay2/d68dfafbabc26b136164d7543fe7add327e4a73104033cbb2dca16483b72b9d0/diff:/var/lib/docker/overlay2/5905329e42bd12e89c18e839ed5ff6da0fc6a8b5d9387648bda0627d0bafde6a/diff:/var/lib/docker/overlay2/82145a29e9a384159c623a66291f2b69cb03d68903a79d930cffd2d0956f9752/diff:/var/lib/docker/overlay2/b00e95aefbd7dd68dc425970c0b16b9f0fb9d1ee26580d51ed056ae4b0f4aead/diff:/var/lib/docker/overlay2/e53b2f03e172d777e4ff1e6eb08d454ed8751058c6702dcfacdde06dee376818/diff:/var/lib/docker/overlay2/c8d5a319c9167c02011b8ca6db83e4a7ad22d0b2cebd1819f233b09aa5226f82/diff:/var/lib/docker/overlay2/7c6b3264dd51ead1fcdd04d8dcea346fa70f7d4a1c8dcb0d33106ff052595f5c/diff:/var/lib/docker/overlay2/50a0c0f4745b50aed3d6ca091293bfc17b9f5004faa032f42f8ac1866ce6fe88/diff:/var/lib/docker/overlay2/6600eeaa45fde3d8981dc097cb3b8fc17ff79bfa6446a17f73115d1bed0
265d8/diff:/var/lib/docker/overlay2/74aababe67d497a154972e099839adfd432c0700905746774b963c3de44570c0/diff:/var/lib/docker/overlay2/a1a90df6b3a27b96f316ef79589896ac8bf703930418b96315d3dae30966611c/diff:/var/lib/docker/overlay2/c4f3d29e56e7de1d1ae0fff2371e7518556de56e784918b82f0bb120fef57d40/diff:/var/lib/docker/overlay2/0887cc3d0a0308de44b000432ba9c0d9b75e91371d6454a5f9cb8ac92e3683a5/diff:/var/lib/docker/overlay2/e319e490a060fa0a339d37c904873849a47cff4a299c43127acc256097842360/diff:/var/lib/docker/overlay2/b128903a99a6d4eb12a378ca66a733122e0def78cee4b16499c7a6edb1cd1623/diff:/var/lib/docker/overlay2/10b741a6b4bc629e1f2a42a1e35e6cec9b8e72901f5d807401ed818c7a184512/diff:/var/lib/docker/overlay2/0a0a6459ab6ea10c57d1832d058c36601b766315eb6b7aafdb729c584ace8452/diff:/var/lib/docker/overlay2/6a17a3f56d97657c295d12a30f41dc2945ba4934edf215a1636ac9ddfd9a4f36/diff:/var/lib/docker/overlay2/de76bffad45064010d5e78dee55719395be9516c18a3b1eb1aed3ff2fba4e265/diff",
"MergedDir": "/var/lib/docker/overlay2/2b600975cc55b63b3fd9b41576fefb8a21fc4ff6823c2d3736d297507e06a0fb/merged",
"UpperDir": "/var/lib/docker/overlay2/2b600975cc55b63b3fd9b41576fefb8a21fc4ff6823c2d3736d297507e06a0fb/diff",
"WorkDir": "/var/lib/docker/overlay2/2b600975cc55b63b3fd9b41576fefb8a21fc4ff6823c2d3736d297507e06a0fb/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-20220630125818-3890",
"Source": "/var/lib/docker/volumes/old-k8s-version-20220630125818-3890/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-20220630125818-3890",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-20220630125818-3890",
"name.minikube.sigs.k8s.io": "old-k8s-version-20220630125818-3890",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "8d77a61fa9c6d20a5ee2f5c02529e2e837fc8b1ccf13bd60ea420c18615bcfbb",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "61111"
}
],
"2376/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "61112"
}
],
"32443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "61113"
}
],
"5000/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "61114"
}
],
"8443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "61110"
}
]
},
"SandboxKey": "/var/run/docker/netns/8d77a61fa9c6",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-20220630125818-3890": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": [
"aa122ad07739",
"old-k8s-version-20220630125818-3890"
],
"NetworkID": "680a3d7490e6c8ea839b29fbdfc4bea8964c4a2befd421f3a9f22738882fa173",
"EndpointID": "52b1df8fac9ba54e264acf2a33dd59a782f39f806c73c26258cb385375fb40dc",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-20220630125818-3890 -n old-k8s-version-20220630125818-3890
E0630 13:02:29.946105 3890 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/.minikube/profiles/cilium-20220630124057-3890/client.crt: no such file or directory
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-20220630125818-3890 -n old-k8s-version-20220630125818-3890: exit status 6 (478.729859ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0630 13:02:30.058988 19309 status.go:413] kubeconfig endpoint: extract IP: "old-k8s-version-20220630125818-3890" does not appear in /Users/jenkins/minikube-integration/darwin-amd64-docker--14420-2699-a1c71987949a080ebcb9d092f1fe4c348bd11abd/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-20220630125818-3890" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (251.55s)