=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-479000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-479000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m17.253537201s)
-- stdout --
* [old-k8s-version-479000] minikube v1.31.2 on Darwin 14.0
- MINIKUBE_LOCATION=17488
- KUBECONFIG=/Users/jenkins/minikube-integration/17488-64832/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/17488-64832/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-479000 in cluster old-k8s-version-479000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 24.0.6 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I1025 18:32:53.979363 80988 out.go:296] Setting OutFile to fd 1 ...
I1025 18:32:53.979645 80988 out.go:343] TERM=,COLORTERM=, which probably does not support color
I1025 18:32:53.979651 80988 out.go:309] Setting ErrFile to fd 2...
I1025 18:32:53.979655 80988 out.go:343] TERM=,COLORTERM=, which probably does not support color
I1025 18:32:53.979832 80988 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17488-64832/.minikube/bin
I1025 18:32:53.981302 80988 out.go:303] Setting JSON to false
I1025 18:32:54.003332 80988 start.go:128] hostinfo: {"hostname":"MacOS-Agent-3.local","uptime":34341,"bootTime":1698249632,"procs":500,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bd1c05a8-24a6-5973-aa69-f3c7c66a87ce"}
W1025 18:32:54.003439 80988 start.go:136] gopshost.Virtualization returned error: not implemented yet
I1025 18:32:54.024871 80988 out.go:177] * [old-k8s-version-479000] minikube v1.31.2 on Darwin 14.0
I1025 18:32:54.067307 80988 out.go:177] - MINIKUBE_LOCATION=17488
I1025 18:32:54.067394 80988 notify.go:220] Checking for updates...
I1025 18:32:54.110285 80988 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/17488-64832/kubeconfig
I1025 18:32:54.133533 80988 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I1025 18:32:54.155448 80988 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1025 18:32:54.176275 80988 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17488-64832/.minikube
I1025 18:32:54.197266 80988 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I1025 18:32:54.218783 80988 config.go:182] Loaded profile config "kubenet-143000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.3
I1025 18:32:54.218905 80988 driver.go:378] Setting default libvirt URI to qemu:///system
I1025 18:32:54.275714 80988 docker.go:122] docker version: linux-24.0.6:Docker Desktop 4.24.2 (124339)
I1025 18:32:54.275852 80988 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1025 18:32:54.378630 80988 info.go:266] docker info: {ID:c18f23ef-4e44-410e-b2ce-38db72a58e15 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:55 OomKillDisable:false NGoroutines:70 SystemTime:2023-10-26 01:32:54.367453155 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:9 KernelVersion:6.4.16-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe
rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6227828736 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.6 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8165feabfdfe38c65b599c4993d227328c231fca Expected:8165feabfdfe38c65b599c4993d227328c231fca} RuncCommit:{ID:v1.1.8-0-g82f18fe Expected:v1.1.8-0-g82f18fe} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconfin
ed name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.11.2-desktop.5] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.22.0-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manage
s Docker extensions Vendor:Docker Inc. Version:v0.2.20] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.8] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Sc
out Vendor:Docker Inc. Version:v1.0.7]] Warnings:<nil>}}
I1025 18:32:54.422094 80988 out.go:177] * Using the docker driver based on user configuration
I1025 18:32:54.443058 80988 start.go:298] selected driver: docker
I1025 18:32:54.443080 80988 start.go:902] validating driver "docker" against <nil>
I1025 18:32:54.443095 80988 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1025 18:32:54.447086 80988 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1025 18:32:54.549787 80988 info.go:266] docker info: {ID:c18f23ef-4e44-410e-b2ce-38db72a58e15 Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:55 OomKillDisable:false NGoroutines:70 SystemTime:2023-10-26 01:32:54.537093247 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:9 KernelVersion:6.4.16-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe
rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6227828736 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.6 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8165feabfdfe38c65b599c4993d227328c231fca Expected:8165feabfdfe38c65b599c4993d227328c231fca} RuncCommit:{ID:v1.1.8-0-g82f18fe Expected:v1.1.8-0-g82f18fe} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconfin
ed name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.11.2-desktop.5] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.22.0-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manage
s Docker extensions Vendor:Docker Inc. Version:v0.2.20] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.8] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Sc
out Vendor:Docker Inc. Version:v1.0.7]] Warnings:<nil>}}
I1025 18:32:54.549958 80988 start_flags.go:309] no existing cluster config was found, will generate one from the flags
I1025 18:32:54.550188 80988 start_flags.go:926] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I1025 18:32:54.571457 80988 out.go:177] * Using Docker Desktop driver with root privileges
I1025 18:32:54.592355 80988 cni.go:84] Creating CNI manager for ""
I1025 18:32:54.592397 80988 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I1025 18:32:54.592422 80988 start_flags.go:323] config:
{Name:old-k8s-version-479000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-479000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:}
I1025 18:32:54.614340 80988 out.go:177] * Starting control plane node old-k8s-version-479000 in cluster old-k8s-version-479000
I1025 18:32:54.636320 80988 cache.go:121] Beginning downloading kic base image for docker with docker
I1025 18:32:54.657104 80988 out.go:177] * Pulling base image ...
I1025 18:32:54.699349 80988 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I1025 18:32:54.699417 80988 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I1025 18:32:54.699423 80988 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 in local docker daemon
I1025 18:32:54.699438 80988 cache.go:56] Caching tarball of preloaded images
I1025 18:32:54.699650 80988 preload.go:174] Found /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I1025 18:32:54.699667 80988 cache.go:59] Finished verifying existence of preloaded tar for v1.16.0 on docker
I1025 18:32:54.699844 80988 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/config.json ...
I1025 18:32:54.700458 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/config.json: {Name:mkad96154b9248eaa3f03a907cda0bd73596ed1c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:32:54.751591 80988 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 in local docker daemon, skipping pull
I1025 18:32:54.751610 80988 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 exists in daemon, skipping load
I1025 18:32:54.751631 80988 cache.go:194] Successfully downloaded all kic artifacts
I1025 18:32:54.751699 80988 start.go:365] acquiring machines lock for old-k8s-version-479000: {Name:mkc5126e3d24e31e0188d7ef4b9443b2bdba7109 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1025 18:32:54.751848 80988 start.go:369] acquired machines lock for "old-k8s-version-479000" in 135.151µs
I1025 18:32:54.751877 80988 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-479000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-479000 Namespace:default APIServer
Name:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:f
alse DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I1025 18:32:54.751969 80988 start.go:125] createHost starting for "" (driver="docker")
I1025 18:32:54.773435 80988 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I1025 18:32:54.773795 80988 start.go:159] libmachine.API.Create for "old-k8s-version-479000" (driver="docker")
I1025 18:32:54.773844 80988 client.go:168] LocalClient.Create starting
I1025 18:32:54.774017 80988 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem
I1025 18:32:54.774095 80988 main.go:141] libmachine: Decoding PEM data...
I1025 18:32:54.774129 80988 main.go:141] libmachine: Parsing certificate...
I1025 18:32:54.774222 80988 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/cert.pem
I1025 18:32:54.774280 80988 main.go:141] libmachine: Decoding PEM data...
I1025 18:32:54.774296 80988 main.go:141] libmachine: Parsing certificate...
I1025 18:32:54.794472 80988 cli_runner.go:164] Run: docker network inspect old-k8s-version-479000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W1025 18:32:54.845615 80988 cli_runner.go:211] docker network inspect old-k8s-version-479000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I1025 18:32:54.845713 80988 network_create.go:281] running [docker network inspect old-k8s-version-479000] to gather additional debugging logs...
I1025 18:32:54.845730 80988 cli_runner.go:164] Run: docker network inspect old-k8s-version-479000
W1025 18:32:54.896634 80988 cli_runner.go:211] docker network inspect old-k8s-version-479000 returned with exit code 1
I1025 18:32:54.896659 80988 network_create.go:284] error running [docker network inspect old-k8s-version-479000]: docker network inspect old-k8s-version-479000: exit status 1
stdout:
[]
stderr:
Error response from daemon: network old-k8s-version-479000 not found
I1025 18:32:54.896683 80988 network_create.go:286] output of [docker network inspect old-k8s-version-479000]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network old-k8s-version-479000 not found
** /stderr **
I1025 18:32:54.896828 80988 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1025 18:32:54.949310 80988 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I1025 18:32:54.949705 80988 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc002424140}
I1025 18:32:54.949721 80988 network_create.go:124] attempt to create docker network old-k8s-version-479000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...
I1025 18:32:54.949784 80988 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-479000 old-k8s-version-479000
W1025 18:32:54.999740 80988 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-479000 old-k8s-version-479000 returned with exit code 1
W1025 18:32:54.999777 80988 network_create.go:149] failed to create docker network old-k8s-version-479000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 65535: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-479000 old-k8s-version-479000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W1025 18:32:54.999795 80988 network_create.go:116] failed to create docker network old-k8s-version-479000 192.168.58.0/24, will retry: subnet is taken
I1025 18:32:55.001220 80988 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I1025 18:32:55.001599 80988 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc002343af0}
I1025 18:32:55.001610 80988 network_create.go:124] attempt to create docker network old-k8s-version-479000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...
I1025 18:32:55.001674 80988 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-479000 old-k8s-version-479000
I1025 18:32:55.088993 80988 network_create.go:108] docker network old-k8s-version-479000 192.168.67.0/24 created
I1025 18:32:55.089042 80988 kic.go:118] calculated static IP "192.168.67.2" for the "old-k8s-version-479000" container
I1025 18:32:55.089168 80988 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I1025 18:32:55.142003 80988 cli_runner.go:164] Run: docker volume create old-k8s-version-479000 --label name.minikube.sigs.k8s.io=old-k8s-version-479000 --label created_by.minikube.sigs.k8s.io=true
I1025 18:32:55.197980 80988 oci.go:103] Successfully created a docker volume old-k8s-version-479000
I1025 18:32:55.198102 80988 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-479000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-479000 --entrypoint /usr/bin/test -v old-k8s-version-479000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 -d /var/lib
I1025 18:32:55.644322 80988 oci.go:107] Successfully prepared a docker volume old-k8s-version-479000
I1025 18:32:55.644360 80988 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I1025 18:32:55.644372 80988 kic.go:191] Starting extracting preloaded images to volume ...
I1025 18:32:55.644469 80988 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-479000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 -I lz4 -xf /preloaded.tar -C /extractDir
I1025 18:32:58.169065 80988 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-479000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 -I lz4 -xf /preloaded.tar -C /extractDir: (2.524449431s)
I1025 18:32:58.169089 80988 kic.go:200] duration metric: took 2.524639 seconds to extract preloaded images to volume
I1025 18:32:58.169213 80988 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I1025 18:32:58.272952 80988 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-479000 --name old-k8s-version-479000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-479000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-479000 --network old-k8s-version-479000 --ip 192.168.67.2 --volume old-k8s-version-479000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883
I1025 18:32:58.561530 80988 cli_runner.go:164] Run: docker container inspect old-k8s-version-479000 --format={{.State.Running}}
I1025 18:32:58.617218 80988 cli_runner.go:164] Run: docker container inspect old-k8s-version-479000 --format={{.State.Status}}
I1025 18:32:58.676878 80988 cli_runner.go:164] Run: docker exec old-k8s-version-479000 stat /var/lib/dpkg/alternatives/iptables
I1025 18:32:58.790568 80988 oci.go:144] the created container "old-k8s-version-479000" has a running status.
I1025 18:32:58.790612 80988 kic.go:222] Creating ssh key for kic: /Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa...
I1025 18:32:59.232360 80988 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I1025 18:32:59.296039 80988 cli_runner.go:164] Run: docker container inspect old-k8s-version-479000 --format={{.State.Status}}
I1025 18:32:59.350757 80988 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I1025 18:32:59.350777 80988 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-479000 chown docker:docker /home/docker/.ssh/authorized_keys]
I1025 18:32:59.446351 80988 cli_runner.go:164] Run: docker container inspect old-k8s-version-479000 --format={{.State.Status}}
I1025 18:32:59.497866 80988 machine.go:88] provisioning docker machine ...
I1025 18:32:59.497908 80988 ubuntu.go:169] provisioning hostname "old-k8s-version-479000"
I1025 18:32:59.498012 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:32:59.549592 80988 main.go:141] libmachine: Using SSH client type: native
I1025 18:32:59.549922 80988 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f54a0] 0x13f8180 <nil> [] 0s} 127.0.0.1 59751 <nil> <nil>}
I1025 18:32:59.549947 80988 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-479000 && echo "old-k8s-version-479000" | sudo tee /etc/hostname
I1025 18:32:59.682274 80988 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-479000
I1025 18:32:59.682392 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:32:59.734242 80988 main.go:141] libmachine: Using SSH client type: native
I1025 18:32:59.734545 80988 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f54a0] 0x13f8180 <nil> [] 0s} 127.0.0.1 59751 <nil> <nil>}
I1025 18:32:59.734558 80988 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-479000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-479000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-479000' | sudo tee -a /etc/hosts;
fi
fi
I1025 18:32:59.855458 80988 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1025 18:32:59.855479 80988 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/17488-64832/.minikube CaCertPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17488-64832/.minikube}
I1025 18:32:59.855502 80988 ubuntu.go:177] setting up certificates
I1025 18:32:59.855509 80988 provision.go:83] configureAuth start
I1025 18:32:59.855584 80988 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-479000
I1025 18:32:59.907222 80988 provision.go:138] copyHostCerts
I1025 18:32:59.907308 80988 exec_runner.go:144] found /Users/jenkins/minikube-integration/17488-64832/.minikube/cert.pem, removing ...
I1025 18:32:59.907317 80988 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17488-64832/.minikube/cert.pem
I1025 18:32:59.907442 80988 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17488-64832/.minikube/cert.pem (1123 bytes)
I1025 18:32:59.907646 80988 exec_runner.go:144] found /Users/jenkins/minikube-integration/17488-64832/.minikube/key.pem, removing ...
I1025 18:32:59.907652 80988 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17488-64832/.minikube/key.pem
I1025 18:32:59.907733 80988 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17488-64832/.minikube/key.pem (1679 bytes)
I1025 18:32:59.907872 80988 exec_runner.go:144] found /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.pem, removing ...
I1025 18:32:59.907878 80988 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.pem
I1025 18:32:59.908421 80988 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.pem (1078 bytes)
I1025 18:32:59.908562 80988 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-479000 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-479000]
I1025 18:33:00.128955 80988 provision.go:172] copyRemoteCerts
I1025 18:33:00.129011 80988 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1025 18:33:00.129078 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:00.181469 80988 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59751 SSHKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa Username:docker}
I1025 18:33:00.271326 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I1025 18:33:00.294845 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I1025 18:33:00.317839 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I1025 18:33:00.341542 80988 provision.go:86] duration metric: configureAuth took 486.00167ms
I1025 18:33:00.341558 80988 ubuntu.go:193] setting minikube options for container-runtime
I1025 18:33:00.341695 80988 config.go:182] Loaded profile config "old-k8s-version-479000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I1025 18:33:00.341768 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:00.393548 80988 main.go:141] libmachine: Using SSH client type: native
I1025 18:33:00.393847 80988 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f54a0] 0x13f8180 <nil> [] 0s} 127.0.0.1 59751 <nil> <nil>}
I1025 18:33:00.393860 80988 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I1025 18:33:00.517513 80988 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I1025 18:33:00.517529 80988 ubuntu.go:71] root file system type: overlay
I1025 18:33:00.517629 80988 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I1025 18:33:00.517717 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:00.568427 80988 main.go:141] libmachine: Using SSH client type: native
I1025 18:33:00.568769 80988 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f54a0] 0x13f8180 <nil> [] 0s} 127.0.0.1 59751 <nil> <nil>}
I1025 18:33:00.568826 80988 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I1025 18:33:00.701772 80988 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I1025 18:33:00.701888 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:00.753851 80988 main.go:141] libmachine: Using SSH client type: native
I1025 18:33:00.754134 80988 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f54a0] 0x13f8180 <nil> [] 0s} 127.0.0.1 59751 <nil> <nil>}
I1025 18:33:00.754148 80988 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I1025 18:33:01.368442 80988 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-09-04 12:30:15.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-10-26 01:33:00.698872182 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I1025 18:33:01.368473 80988 machine.go:91] provisioned docker machine in 1.87052983s
I1025 18:33:01.368480 80988 client.go:171] LocalClient.Create took 6.594431418s
I1025 18:33:01.368495 80988 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-479000" took 6.594505758s
I1025 18:33:01.368502 80988 start.go:300] post-start starting for "old-k8s-version-479000" (driver="docker")
I1025 18:33:01.368512 80988 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1025 18:33:01.368580 80988 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1025 18:33:01.368643 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:01.421509 80988 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59751 SSHKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa Username:docker}
I1025 18:33:01.511333 80988 ssh_runner.go:195] Run: cat /etc/os-release
I1025 18:33:01.515805 80988 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1025 18:33:01.515829 80988 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I1025 18:33:01.515837 80988 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I1025 18:33:01.515847 80988 info.go:137] Remote host: Ubuntu 22.04.3 LTS
I1025 18:33:01.515859 80988 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17488-64832/.minikube/addons for local assets ...
I1025 18:33:01.515951 80988 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17488-64832/.minikube/files for local assets ...
I1025 18:33:01.516122 80988 filesync.go:149] local asset: /Users/jenkins/minikube-integration/17488-64832/.minikube/files/etc/ssl/certs/652922.pem -> 652922.pem in /etc/ssl/certs
I1025 18:33:01.516307 80988 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I1025 18:33:01.525398 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/files/etc/ssl/certs/652922.pem --> /etc/ssl/certs/652922.pem (1708 bytes)
I1025 18:33:01.548314 80988 start.go:303] post-start completed in 179.798928ms
I1025 18:33:01.548849 80988 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-479000
I1025 18:33:01.600673 80988 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/config.json ...
I1025 18:33:01.601094 80988 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1025 18:33:01.601155 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:01.653328 80988 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59751 SSHKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa Username:docker}
I1025 18:33:01.741822 80988 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1025 18:33:01.747350 80988 start.go:128] duration metric: createHost completed in 6.995155783s
I1025 18:33:01.747370 80988 start.go:83] releasing machines lock for "old-k8s-version-479000", held for 6.995302968s
I1025 18:33:01.747448 80988 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-479000
I1025 18:33:01.798891 80988 ssh_runner.go:195] Run: cat /version.json
I1025 18:33:01.798934 80988 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I1025 18:33:01.798957 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:01.799001 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:01.859505 80988 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59751 SSHKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa Username:docker}
I1025 18:33:01.859604 80988 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59751 SSHKeyPath:/Users/jenkins/minikube-integration/17488-64832/.minikube/machines/old-k8s-version-479000/id_rsa Username:docker}
I1025 18:33:01.945387 80988 ssh_runner.go:195] Run: systemctl --version
I1025 18:33:02.051731 80988 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I1025 18:33:02.057734 80988 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I1025 18:33:02.083223 80988 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I1025 18:33:02.083319 80988 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I1025 18:33:02.100820 80988 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I1025 18:33:02.117915 80988 cni.go:308] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I1025 18:33:02.117937 80988 start.go:472] detecting cgroup driver to use...
I1025 18:33:02.117955 80988 detect.go:196] detected "cgroupfs" cgroup driver on host os
I1025 18:33:02.118072 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I1025 18:33:02.134671 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.1"|' /etc/containerd/config.toml"
I1025 18:33:02.145411 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I1025 18:33:02.156055 80988 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I1025 18:33:02.156133 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I1025 18:33:02.167719 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1025 18:33:02.178431 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I1025 18:33:02.189889 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1025 18:33:02.200871 80988 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I1025 18:33:02.211246 80988 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I1025 18:33:02.222154 80988 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I1025 18:33:02.231580 80988 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I1025 18:33:02.240824 80988 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1025 18:33:02.304777 80988 ssh_runner.go:195] Run: sudo systemctl restart containerd
I1025 18:33:02.395616 80988 start.go:472] detecting cgroup driver to use...
I1025 18:33:02.395635 80988 detect.go:196] detected "cgroupfs" cgroup driver on host os
I1025 18:33:02.395698 80988 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I1025 18:33:02.414282 80988 cruntime.go:279] skipping containerd shutdown because we are bound to it
I1025 18:33:02.414377 80988 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I1025 18:33:02.426896 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I1025 18:33:02.446457 80988 ssh_runner.go:195] Run: which cri-dockerd
I1025 18:33:02.451682 80988 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I1025 18:33:02.462541 80988 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I1025 18:33:02.483856 80988 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I1025 18:33:02.573208 80988 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I1025 18:33:02.670329 80988 docker.go:555] configuring docker to use "cgroupfs" as cgroup driver...
I1025 18:33:02.670434 80988 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I1025 18:33:02.688992 80988 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1025 18:33:02.771595 80988 ssh_runner.go:195] Run: sudo systemctl restart docker
I1025 18:33:03.026436 80988 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I1025 18:33:03.051902 80988 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I1025 18:33:03.123333 80988 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 24.0.6 ...
I1025 18:33:03.123420 80988 cli_runner.go:164] Run: docker exec -t old-k8s-version-479000 dig +short host.docker.internal
I1025 18:33:03.246721 80988 network.go:96] got host ip for mount in container by digging dns: 192.168.65.254
I1025 18:33:03.246805 80988 ssh_runner.go:195] Run: grep 192.168.65.254 host.minikube.internal$ /etc/hosts
I1025 18:33:03.251926 80988 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.254 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1025 18:33:03.264351 80988 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-479000
I1025 18:33:03.316038 80988 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I1025 18:33:03.316107 80988 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I1025 18:33:03.338721 80988 docker.go:693] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I1025 18:33:03.338736 80988 docker.go:699] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I1025 18:33:03.338796 80988 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I1025 18:33:03.348908 80988 ssh_runner.go:195] Run: which lz4
I1025 18:33:03.354371 80988 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I1025 18:33:03.359519 80988 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I1025 18:33:03.359560 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (369789069 bytes)
I1025 18:33:09.124365 80988 docker.go:657] Took 5.769873 seconds to copy over tarball
I1025 18:33:09.124435 80988 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I1025 18:33:11.575908 80988 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.451383193s)
I1025 18:33:11.575924 80988 ssh_runner.go:146] rm: /preloaded.tar.lz4
I1025 18:33:11.627864 80988 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I1025 18:33:11.638675 80988 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2499 bytes)
I1025 18:33:11.661138 80988 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1025 18:33:11.727833 80988 ssh_runner.go:195] Run: sudo systemctl restart docker
I1025 18:33:12.361002 80988 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I1025 18:33:12.382479 80988 docker.go:693] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I1025 18:33:12.382494 80988 docker.go:699] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I1025 18:33:12.382505 80988 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.16.0 registry.k8s.io/kube-controller-manager:v1.16.0 registry.k8s.io/kube-scheduler:v1.16.0 registry.k8s.io/kube-proxy:v1.16.0 registry.k8s.io/pause:3.1 registry.k8s.io/etcd:3.3.15-0 registry.k8s.io/coredns:1.6.2 gcr.io/k8s-minikube/storage-provisioner:v5]
I1025 18:33:12.390682 80988 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.16.0
I1025 18:33:12.390769 80988 image.go:134] retrieving image: registry.k8s.io/coredns:1.6.2
I1025 18:33:12.390800 80988 image.go:134] retrieving image: registry.k8s.io/etcd:3.3.15-0
I1025 18:33:12.390834 80988 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I1025 18:33:12.391321 80988 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.16.0
I1025 18:33:12.391506 80988 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.16.0
I1025 18:33:12.392435 80988 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.16.0
I1025 18:33:12.392499 80988 image.go:134] retrieving image: registry.k8s.io/pause:3.1
I1025 18:33:12.400279 80988 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.16.0
I1025 18:33:12.400706 80988 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.16.0
I1025 18:33:12.401964 80988 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I1025 18:33:12.402131 80988 image.go:177] daemon lookup for registry.k8s.io/etcd:3.3.15-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.3.15-0
I1025 18:33:12.402048 80988 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.16.0
I1025 18:33:12.402415 80988 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.16.0
I1025 18:33:12.402386 80988 image.go:177] daemon lookup for registry.k8s.io/coredns:1.6.2: Error response from daemon: No such image: registry.k8s.io/coredns:1.6.2
I1025 18:33:12.402596 80988 image.go:177] daemon lookup for registry.k8s.io/pause:3.1: Error response from daemon: No such image: registry.k8s.io/pause:3.1
I1025 18:33:13.088923 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.16.0
I1025 18:33:13.111990 80988 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.16.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.16.0" does not exist at hash "b305571ca60a5a7818bda47da122683d75e8a1907475681ee8b1efbd06bff12e" in container runtime
I1025 18:33:13.112034 80988 docker.go:318] Removing image: registry.k8s.io/kube-apiserver:v1.16.0
I1025 18:33:13.112085 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-apiserver:v1.16.0
I1025 18:33:13.134360 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.16.0
I1025 18:33:13.213782 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.16.0
I1025 18:33:13.236364 80988 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.16.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.16.0" does not exist at hash "06a629a7e51cdcc81a5ed6a3e6650348312f20c954ac52ee489a023628ec9c7d" in container runtime
I1025 18:33:13.236399 80988 docker.go:318] Removing image: registry.k8s.io/kube-controller-manager:v1.16.0
I1025 18:33:13.236468 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-controller-manager:v1.16.0
I1025 18:33:13.258674 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.16.0
I1025 18:33:13.877939 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/etcd:3.3.15-0
I1025 18:33:13.901265 80988 cache_images.go:116] "registry.k8s.io/etcd:3.3.15-0" needs transfer: "registry.k8s.io/etcd:3.3.15-0" does not exist at hash "b2756210eeabf84f3221da9959e9483f3919dc2aaab4cd45e7cd072fcbde27ed" in container runtime
I1025 18:33:13.901294 80988 docker.go:318] Removing image: registry.k8s.io/etcd:3.3.15-0
I1025 18:33:13.901356 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/etcd:3.3.15-0
I1025 18:33:13.924368 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.3.15-0
I1025 18:33:13.976430 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I1025 18:33:14.190379 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.16.0
I1025 18:33:14.213036 80988 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.16.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.16.0" does not exist at hash "301ddc62b80b16315d3c2653cf3888370394277afb3187614cfa20edc352ca0a" in container runtime
I1025 18:33:14.213078 80988 docker.go:318] Removing image: registry.k8s.io/kube-scheduler:v1.16.0
I1025 18:33:14.213148 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-scheduler:v1.16.0
I1025 18:33:14.235261 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.16.0
I1025 18:33:14.489632 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/coredns:1.6.2
I1025 18:33:14.512441 80988 cache_images.go:116] "registry.k8s.io/coredns:1.6.2" needs transfer: "registry.k8s.io/coredns:1.6.2" does not exist at hash "bf261d157914477ee1a5969d28ec687f3fbfc9fb5a664b22df78e57023b0e03b" in container runtime
I1025 18:33:14.512481 80988 docker.go:318] Removing image: registry.k8s.io/coredns:1.6.2
I1025 18:33:14.512546 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/coredns:1.6.2
I1025 18:33:14.534925 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.2
I1025 18:33:14.803497 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.16.0
I1025 18:33:14.825704 80988 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.16.0" needs transfer: "registry.k8s.io/kube-proxy:v1.16.0" does not exist at hash "c21b0c7400f988db4777858edd13b6d3930d62d7ccf026d2415485a52037f384" in container runtime
I1025 18:33:14.825732 80988 docker.go:318] Removing image: registry.k8s.io/kube-proxy:v1.16.0
I1025 18:33:14.825778 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-proxy:v1.16.0
I1025 18:33:14.847147 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.16.0
I1025 18:33:15.122463 80988 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/pause:3.1
I1025 18:33:15.145360 80988 cache_images.go:116] "registry.k8s.io/pause:3.1" needs transfer: "registry.k8s.io/pause:3.1" does not exist at hash "da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e" in container runtime
I1025 18:33:15.145385 80988 docker.go:318] Removing image: registry.k8s.io/pause:3.1
I1025 18:33:15.145452 80988 ssh_runner.go:195] Run: docker rmi registry.k8s.io/pause:3.1
I1025 18:33:15.166320 80988 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1
I1025 18:33:15.166366 80988 cache_images.go:92] LoadImages completed in 2.783766426s
W1025 18:33:15.166418 80988 out.go:239] X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.16.0: no such file or directory
X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/17488-64832/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.16.0: no such file or directory
I1025 18:33:15.166488 80988 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I1025 18:33:15.220589 80988 cni.go:84] Creating CNI manager for ""
I1025 18:33:15.220607 80988 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I1025 18:33:15.220631 80988 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I1025 18:33:15.220661 80988 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-479000 NodeName:old-k8s-version-479000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I1025 18:33:15.220780 80988 kubeadm.go:181] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-479000"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-479000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I1025 18:33:15.220858 80988 kubeadm.go:976] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-479000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-479000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I1025 18:33:15.220919 80988 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I1025 18:33:15.230738 80988 binaries.go:44] Found k8s binaries, skipping transfer
I1025 18:33:15.230811 80988 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I1025 18:33:15.240538 80988 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I1025 18:33:15.258182 80988 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I1025 18:33:15.275835 80988 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I1025 18:33:15.293527 80988 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I1025 18:33:15.298184 80988 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1025 18:33:15.310041 80988 certs.go:56] Setting up /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000 for IP: 192.168.67.2
I1025 18:33:15.310062 80988 certs.go:190] acquiring lock for shared ca certs: {Name:mk3b233645537eeaa35f16b83a4ace6d87ff2e20 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.310229 80988 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.key
I1025 18:33:15.310289 80988 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/17488-64832/.minikube/proxy-client-ca.key
I1025 18:33:15.310330 80988 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.key
I1025 18:33:15.310344 80988 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.crt with IP's: []
I1025 18:33:15.711923 80988 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.crt ...
I1025 18:33:15.711941 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.crt: {Name:mk75e02a0990e49d98d16d9761310f8c3b274942 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.712251 80988 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.key ...
I1025 18:33:15.712259 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/client.key: {Name:mk5e03f62ffc15bf5973af58efc90cd11c9751ad Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.712493 80988 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key.c7fa3a9e
I1025 18:33:15.712510 80988 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt.c7fa3a9e with IP's: [192.168.67.2 10.96.0.1 127.0.0.1 10.0.0.1]
I1025 18:33:15.804350 80988 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt.c7fa3a9e ...
I1025 18:33:15.804362 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt.c7fa3a9e: {Name:mke547eb8324beb02766740c6c5c7b0719ec7acf Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.804624 80988 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key.c7fa3a9e ...
I1025 18:33:15.804632 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key.c7fa3a9e: {Name:mk2badd7d7ba697f88806230120ac20ba374f3b3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.804844 80988 certs.go:337] copying /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt.c7fa3a9e -> /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt
I1025 18:33:15.805027 80988 certs.go:341] copying /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key.c7fa3a9e -> /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key
I1025 18:33:15.805191 80988 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.key
I1025 18:33:15.805205 80988 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.crt with IP's: []
I1025 18:33:15.920394 80988 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.crt ...
I1025 18:33:15.920405 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.crt: {Name:mk675b7160007201bc9e59a8a61dc1772333de4f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.920678 80988 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.key ...
I1025 18:33:15.920686 80988 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.key: {Name:mk47bc066efb1e7a50468dedabd2cd9dc4be8b8a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1025 18:33:15.921073 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/65292.pem (1338 bytes)
W1025 18:33:15.921127 80988 certs.go:433] ignoring /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/65292_empty.pem, impossibly tiny 0 bytes
I1025 18:33:15.921139 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca-key.pem (1679 bytes)
I1025 18:33:15.921173 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/ca.pem (1078 bytes)
I1025 18:33:15.921205 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/cert.pem (1123 bytes)
I1025 18:33:15.921237 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/certs/key.pem (1679 bytes)
I1025 18:33:15.921305 80988 certs.go:437] found cert: /Users/jenkins/minikube-integration/17488-64832/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/17488-64832/.minikube/files/etc/ssl/certs/652922.pem (1708 bytes)
I1025 18:33:15.921856 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I1025 18:33:15.946736 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I1025 18:33:15.970636 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I1025 18:33:15.993917 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/profiles/old-k8s-version-479000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I1025 18:33:16.017541 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I1025 18:33:16.040615 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I1025 18:33:16.064346 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I1025 18:33:16.087506 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I1025 18:33:16.111085 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/certs/65292.pem --> /usr/share/ca-certificates/65292.pem (1338 bytes)
I1025 18:33:16.134348 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/files/etc/ssl/certs/652922.pem --> /usr/share/ca-certificates/652922.pem (1708 bytes)
I1025 18:33:16.157807 80988 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17488-64832/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I1025 18:33:16.181744 80988 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I1025 18:33:16.199562 80988 ssh_runner.go:195] Run: openssl version
I1025 18:33:16.206044 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/65292.pem && ln -fs /usr/share/ca-certificates/65292.pem /etc/ssl/certs/65292.pem"
I1025 18:33:16.216668 80988 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/65292.pem
I1025 18:33:16.221347 80988 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Oct 26 00:44 /usr/share/ca-certificates/65292.pem
I1025 18:33:16.221394 80988 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/65292.pem
I1025 18:33:16.228602 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/65292.pem /etc/ssl/certs/51391683.0"
I1025 18:33:16.239031 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/652922.pem && ln -fs /usr/share/ca-certificates/652922.pem /etc/ssl/certs/652922.pem"
I1025 18:33:16.249807 80988 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/652922.pem
I1025 18:33:16.254266 80988 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Oct 26 00:44 /usr/share/ca-certificates/652922.pem
I1025 18:33:16.254315 80988 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/652922.pem
I1025 18:33:16.261205 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/652922.pem /etc/ssl/certs/3ec20f2e.0"
I1025 18:33:16.271591 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I1025 18:33:16.281772 80988 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I1025 18:33:16.286433 80988 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Oct 26 00:39 /usr/share/ca-certificates/minikubeCA.pem
I1025 18:33:16.286498 80988 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I1025 18:33:16.294007 80988 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I1025 18:33:16.304154 80988 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd
I1025 18:33:16.308677 80988 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2
stdout:
stderr:
ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory
I1025 18:33:16.308724 80988 kubeadm.go:404] StartCluster: {Name:old-k8s-version-479000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-479000 Namespace:default APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disab
leMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s GPUs:}
I1025 18:33:16.308832 80988 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I1025 18:33:16.329066 80988 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I1025 18:33:16.338945 80988 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I1025 18:33:16.348479 80988 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I1025 18:33:16.348535 80988 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I1025 18:33:16.358198 80988 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I1025 18:33:16.358225 80988 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I1025 18:33:16.410549 80988 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I1025 18:33:16.410592 80988 kubeadm.go:322] [preflight] Running pre-flight checks
I1025 18:33:16.672387 80988 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I1025 18:33:16.672484 80988 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I1025 18:33:16.672568 80988 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I1025 18:33:16.864163 80988 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I1025 18:33:16.864844 80988 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I1025 18:33:16.871966 80988 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I1025 18:33:16.945692 80988 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I1025 18:33:16.970052 80988 out.go:204] - Generating certificates and keys ...
I1025 18:33:16.970137 80988 kubeadm.go:322] [certs] Using existing ca certificate authority
I1025 18:33:16.970241 80988 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I1025 18:33:17.045896 80988 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I1025 18:33:17.218031 80988 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I1025 18:33:17.341384 80988 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I1025 18:33:17.399187 80988 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I1025 18:33:17.577307 80988 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I1025 18:33:17.577419 80988 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I1025 18:33:17.760120 80988 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I1025 18:33:17.760241 80988 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I1025 18:33:18.036580 80988 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I1025 18:33:18.248937 80988 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I1025 18:33:18.447449 80988 kubeadm.go:322] [certs] Generating "sa" key and public key
I1025 18:33:18.447656 80988 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1025 18:33:18.672821 80988 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I1025 18:33:18.888402 80988 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I1025 18:33:18.996950 80988 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1025 18:33:19.156349 80988 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I1025 18:33:19.156878 80988 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I1025 18:33:19.178604 80988 out.go:204] - Booting up control plane ...
I1025 18:33:19.178678 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I1025 18:33:19.178755 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I1025 18:33:19.178821 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I1025 18:33:19.178894 80988 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1025 18:33:19.179045 80988 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I1025 18:33:59.166397 80988 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I1025 18:33:59.167181 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:33:59.167474 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:34:04.167979 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:34:04.168169 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:34:14.169429 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:34:14.169619 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:34:34.172549 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:34:34.172852 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:35:14.174080 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:35:14.174320 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:35:14.174333 80988 kubeadm.go:322]
I1025 18:35:14.174375 80988 kubeadm.go:322] Unfortunately, an error has occurred:
I1025 18:35:14.174417 80988 kubeadm.go:322] timed out waiting for the condition
I1025 18:35:14.174424 80988 kubeadm.go:322]
I1025 18:35:14.174459 80988 kubeadm.go:322] This error is likely caused by:
I1025 18:35:14.174493 80988 kubeadm.go:322] - The kubelet is not running
I1025 18:35:14.174597 80988 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I1025 18:35:14.174606 80988 kubeadm.go:322]
I1025 18:35:14.174703 80988 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I1025 18:35:14.174735 80988 kubeadm.go:322] - 'systemctl status kubelet'
I1025 18:35:14.174769 80988 kubeadm.go:322] - 'journalctl -xeu kubelet'
I1025 18:35:14.174775 80988 kubeadm.go:322]
I1025 18:35:14.174902 80988 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I1025 18:35:14.175034 80988 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I1025 18:35:14.175150 80988 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I1025 18:35:14.175234 80988 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I1025 18:35:14.175352 80988 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I1025 18:35:14.175407 80988 kubeadm.go:322] - 'docker logs CONTAINERID'
I1025 18:35:14.177351 80988 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I1025 18:35:14.177429 80988 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I1025 18:35:14.177528 80988 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
I1025 18:35:14.177618 80988 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I1025 18:35:14.177689 80988 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I1025 18:35:14.177748 80988 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W1025 18:35:14.177832 80988 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-479000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I1025 18:35:14.177863 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I1025 18:35:14.595022 80988 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I1025 18:35:14.607292 80988 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I1025 18:35:14.607349 80988 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I1025 18:35:14.617161 80988 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I1025 18:35:14.617196 80988 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I1025 18:35:14.670227 80988 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I1025 18:35:14.670298 80988 kubeadm.go:322] [preflight] Running pre-flight checks
I1025 18:35:14.922296 80988 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I1025 18:35:14.922375 80988 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I1025 18:35:14.922456 80988 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I1025 18:35:15.106762 80988 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I1025 18:35:15.107638 80988 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I1025 18:35:15.114475 80988 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I1025 18:35:15.189936 80988 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I1025 18:35:15.211612 80988 out.go:204] - Generating certificates and keys ...
I1025 18:35:15.211695 80988 kubeadm.go:322] [certs] Using existing ca certificate authority
I1025 18:35:15.211766 80988 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I1025 18:35:15.211827 80988 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I1025 18:35:15.211871 80988 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I1025 18:35:15.211925 80988 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I1025 18:35:15.212013 80988 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I1025 18:35:15.212072 80988 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I1025 18:35:15.212131 80988 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I1025 18:35:15.212231 80988 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I1025 18:35:15.212322 80988 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I1025 18:35:15.212359 80988 kubeadm.go:322] [certs] Using the existing "sa" key
I1025 18:35:15.212425 80988 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1025 18:35:15.245361 80988 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I1025 18:35:15.409936 80988 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I1025 18:35:15.510588 80988 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1025 18:35:15.598865 80988 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I1025 18:35:15.599660 80988 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I1025 18:35:15.621350 80988 out.go:204] - Booting up control plane ...
I1025 18:35:15.621436 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I1025 18:35:15.621516 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I1025 18:35:15.621579 80988 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I1025 18:35:15.621644 80988 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1025 18:35:15.621774 80988 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I1025 18:35:55.609710 80988 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I1025 18:35:55.610362 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:35:55.610552 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:36:00.612358 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:36:00.612567 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:36:10.614051 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:36:10.614266 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:36:30.616749 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:36:30.616976 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:37:10.619615 80988 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I1025 18:37:10.619996 80988 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I1025 18:37:10.620013 80988 kubeadm.go:322]
I1025 18:37:10.620051 80988 kubeadm.go:322] Unfortunately, an error has occurred:
I1025 18:37:10.620095 80988 kubeadm.go:322] timed out waiting for the condition
I1025 18:37:10.620109 80988 kubeadm.go:322]
I1025 18:37:10.620174 80988 kubeadm.go:322] This error is likely caused by:
I1025 18:37:10.620201 80988 kubeadm.go:322] - The kubelet is not running
I1025 18:37:10.620285 80988 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I1025 18:37:10.620296 80988 kubeadm.go:322]
I1025 18:37:10.620403 80988 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I1025 18:37:10.620434 80988 kubeadm.go:322] - 'systemctl status kubelet'
I1025 18:37:10.620465 80988 kubeadm.go:322] - 'journalctl -xeu kubelet'
I1025 18:37:10.620474 80988 kubeadm.go:322]
I1025 18:37:10.620595 80988 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I1025 18:37:10.620723 80988 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I1025 18:37:10.620826 80988 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I1025 18:37:10.620876 80988 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I1025 18:37:10.620932 80988 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I1025 18:37:10.620962 80988 kubeadm.go:322] - 'docker logs CONTAINERID'
I1025 18:37:10.622942 80988 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I1025 18:37:10.623022 80988 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I1025 18:37:10.623132 80988 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
I1025 18:37:10.623218 80988 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I1025 18:37:10.623300 80988 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I1025 18:37:10.623366 80988 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I1025 18:37:10.623392 80988 kubeadm.go:406] StartCluster complete in 3m54.307641011s
I1025 18:37:10.623485 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I1025 18:37:10.644434 80988 logs.go:284] 0 containers: []
W1025 18:37:10.644448 80988 logs.go:286] No container was found matching "kube-apiserver"
I1025 18:37:10.644520 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I1025 18:37:10.663774 80988 logs.go:284] 0 containers: []
W1025 18:37:10.663788 80988 logs.go:286] No container was found matching "etcd"
I1025 18:37:10.663857 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I1025 18:37:10.684779 80988 logs.go:284] 0 containers: []
W1025 18:37:10.684792 80988 logs.go:286] No container was found matching "coredns"
I1025 18:37:10.684863 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I1025 18:37:10.704572 80988 logs.go:284] 0 containers: []
W1025 18:37:10.704585 80988 logs.go:286] No container was found matching "kube-scheduler"
I1025 18:37:10.704673 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I1025 18:37:10.724210 80988 logs.go:284] 0 containers: []
W1025 18:37:10.724224 80988 logs.go:286] No container was found matching "kube-proxy"
I1025 18:37:10.724291 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I1025 18:37:10.744729 80988 logs.go:284] 0 containers: []
W1025 18:37:10.744742 80988 logs.go:286] No container was found matching "kube-controller-manager"
I1025 18:37:10.744806 80988 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I1025 18:37:10.765613 80988 logs.go:284] 0 containers: []
W1025 18:37:10.765630 80988 logs.go:286] No container was found matching "kindnet"
I1025 18:37:10.765645 80988 logs.go:123] Gathering logs for kubelet ...
I1025 18:37:10.765660 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I1025 18:37:10.804341 80988 logs.go:123] Gathering logs for dmesg ...
I1025 18:37:10.804355 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1025 18:37:10.819151 80988 logs.go:123] Gathering logs for describe nodes ...
I1025 18:37:10.819172 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W1025 18:37:10.877192 80988 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I1025 18:37:10.877205 80988 logs.go:123] Gathering logs for Docker ...
I1025 18:37:10.877212 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I1025 18:37:10.894899 80988 logs.go:123] Gathering logs for container status ...
I1025 18:37:10.894915 80988 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W1025 18:37:10.957012 80988 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W1025 18:37:10.957033 80988 out.go:239] *
*
W1025 18:37:10.957091 80988 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W1025 18:37:10.957107 80988 out.go:239] *
*
W1025 18:37:10.957785 80988 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1025 18:37:11.019607 80988 out.go:177]
W1025 18:37:11.061852 80988 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W1025 18:37:11.061922 80988 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W1025 18:37:11.061961 80988 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I1025 18:37:11.103678 80988 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-479000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-479000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-479000:
-- stdout --
[
{
"Id": "5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69",
"Created": "2023-10-26T01:32:58.324650138Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 308034,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-10-26T01:32:58.551798864Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:3e615aae66792e89a7d2c001b5c02b5e78a999706d53f7c8dbfcff1520487fdd",
"ResolvConfPath": "/var/lib/docker/containers/5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69/hostname",
"HostsPath": "/var/lib/docker/containers/5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69/hosts",
"LogPath": "/var/lib/docker/containers/5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69/5e3f3c28e57cb270f49205eeb37ac08f10551bd5b9480af216c9e9d4af914f69-json.log",
"Name": "/old-k8s-version-479000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-479000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-479000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/38224b4095bfa384a8392fe28fd4684bbed1e685b1da03f4bd770e877c6a5c2b-init/diff:/var/lib/docker/overlay2/d80c3c6ebb3e22fc0994c621eeb60a01efaecbf75cf8c7e33299fa73160e5f82/diff",
"MergedDir": "/var/lib/docker/overlay2/38224b4095bfa384a8392fe28fd4684bbed1e685b1da03f4bd770e877c6a5c2b/merged",
"UpperDir": "/var/lib/docker/overlay2/38224b4095bfa384a8392fe28fd4684bbed1e685b1da03f4bd770e877c6a5c2b/diff",
"WorkDir": "/var/lib/docker/overlay2/38224b4095bfa384a8392fe28fd4684bbed1e685b1da03f4bd770e877c6a5c2b/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-479000",
"Source": "/var/lib/docker/volumes/old-k8s-version-479000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-479000",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1698055645-17423@sha256:fb2566ae68d58d9dce5cb4087954a42bedc9f0c47c18aef3d28a238a8beeb883",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-479000",
"name.minikube.sigs.k8s.io": "old-k8s-version-479000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "61ef09e73509324069dc4e60373e50da7bb9f4aa73cc234fc24dd5fc9d713013",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59751"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59747"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59748"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59749"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59750"
}
]
},
"SandboxKey": "/var/run/docker/netns/61ef09e73509",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-479000": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"5e3f3c28e57c",
"old-k8s-version-479000"
],
"NetworkID": "e1c286b1eee5e63f7c876927f11c7e5f513aa124ea1227ec48978fbb98cbe026",
"EndpointID": "e6325b666370a2ad46be3ad7725f8a9875c0e80e153924797402687b3b6529c1",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-479000 -n old-k8s-version-479000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-479000 -n old-k8s-version-479000: exit status 6 (405.594314ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E1025 18:37:11.670048 82083 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-479000" does not appear in /Users/jenkins/minikube-integration/17488-64832/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-479000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (257.75s)